strtok problem in calling

Question

I have a function using strtok like this

void f1(char *name)
{
...
char *tmp;
tmp = strtok(names, " ,");
while(tmp)
{
...
tmp = strtok(NULL, " ,");
}
...
}

And i have a call f1("abc,def");

Problem is that in first call f1 gets abc,def and in 2nd call gets just abc

I am confused.. Why is this so?

Answer 1

strtok() modifies its input string by overwriting the delimiters with a 0; so, assuming your code looks something like this:

char parm[] = "abc,def";

f1(parm);
f1(parm);

after the first call to f1, the ',' character is overwritten with a 0, which is a string terminator, so the second call only sees "abc" as the string.

Note that because strtok() modifies its input, you do not want to pass it a string literal as an argument; attempting to modify the contents of a string literal invokes undefined behavior.

The safe thing to do is to create a local string within f1 and copy the contents of names to it, then pass that local string to strtok(). The following should work with C99:

void f1(char *name)
{
  size_t len = strlen(name);
  char localstr[len+1];
  char *tmp;
  strcpy(localstr, name);

  tmp = strtok(localstr, " ,");
  while(tmp)
  {
    ...
    tmp = strtok(NULL, " ,");
  }
}

Answer 2

You say:

And i have a call f1("abc,def");

That call is illegal - strtok modifies its first parameter and you are not allowed to modify string literals. What you get is undefined behaviour - anything could happen. You want:

char a[] = "abc,def";
f1( a );

Answer 3

Are you truly passing in a string literal?

f1("abc,def");

will pass a pointer to the string literal to the strtok() in f1() - since strtok() modifies the string, and a string literal can't be modified, so you'll get undefined behavior (though I would expect a crash or fault instead of unexpected results).

Answer 4

strtok puts a null terminator after each token it returns. This means that it destroys the original string: After calling it, your string will be terminated after the first token, resulting in the behavior you see.

To keep the original string unchanged, you will need to make a copy of it before calling strtok.