Trying to tokenize a string [duplicate]

StackOverflow https://stackoverflow.com/questions/12892728

  •  07-07-2021
  •  | 
  •  

Question

Possible Duplicate:
How do I tokenize a string in C++?

I have this text file 

q1 t q2

q2 e q3

q3 x q4

q4 t q5 q6 q11

q5 | q6 q11

I want to extract each element that is separated by a space. For example, in line one I want to be able to extract "q1" "t" and "q2" as separate tokens.

I'm thinking there's two ways to go about this

  1. read from the file, token by token, using ifstream>>. The problem I'm having with this approach is that I don't know how to tell when the end of a line has been reached so that I can move onto the next line.

  2. The other approach, is to get the whole line at once with getline(); The problem with this approach is that I have to tokenize the string myself, and each line is different so I'm not sure that's the best idea. I'm pretty blown away there's no built in way to do this, besides strtok() which looks to be not at all what I want. Thanks guys any help is appreciated.

Was it helpful?

Solution

Use getline, and to tokenize the resulting string, put it into a std::stringstream and extract the tokens from that.

std::string line_string;
while ( getline( file, line_string ) ) {
    std::istringstream line( line_string );
    std::string token;
    while ( line >> token ) {
        do something with token
    }
}

OTHER TIPS

If you only want to use standard library, you can go with getline and then use a stringstream to ifstream>> it word by word.

Otherwise C++ is indeed quite limited in terms of text processing and you could do things more easily with Qt's strings and streams, or string algorithms from boost (including split), which integrate well with STL strings.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top