How to read 1000 or more columns data from file using c/c++ language？

Question

A data file with 10000 rows and 1000 columns. I want to save a entire line to an array or each column to a variant.

There is a standard function fscanf in C. If use this function, I need write the format 1000 times.

fscanf(pFile, "%f,%f,%f,%f,%f,%f,......", &a[0], &a[1],...,a[999]);

It is almost impossible like this when programming in C. But, I have no idea to implement it with C language. Any suggestions or solutions?

And, how to read or extract some of columns data?

Answer 1

Read the file line by line using fgets() into a suitably large buffer. Don't be afraid to use a buffer of 32 KB or something, just to be very sure all the fields fit.

Then parse the line in a loop, perhaps using strtok() or just plain old strtod(). Note that the latter returns a pointer to the first character that was not considered a number; this is where your parsing will continue for the next number. Perhaps you need to add an inner loop to "eat" whitespace (or whatever separators you have).

Answer 2

You could read the file line by line, and then extract the numbers in a loop.