How to use strtok to delimit and get new strings

Question

I am trying to use strtok to break up a string and to store the extracted strings into new arrays so I can use them seperately as commands or something.

#include <stdio.h>
#include <string.h>

main()
{

   char test_string[50]="string to split up";
   char *sub_string;

   /* Extract first string */
   printf("%s\n", strtok(test_string, " "));

   /* Extract remaining strings   */
   while ( (sub_string=strtok(NULL, " ")) != NULL)
   {
       printf("%s\n", sub_string);
   }
}

This prints out the string I am looking for without the spaces, but how would I go about getting each of the words saved into seperate string variables instead of just printing them? Basically I want "string" to be saved into string1[] and "to" to be saved in string2[] and so forth.

Answer 1

The original string gets "chopped into pieces" by the strtok operation, so you can actually do the following:

#include <stdio.h>
#include <string.h>

int main(void)
{

   char test_string[50]="string to split up";
   char *sub_string[50];
   int ii = -1;
   /* Extract first string */
   sub_string[++ii]=strtok(test_string, " "));
   printf("%s\n", sub_string[0]);
   /* Extract remaining strings   */
   while ( (sub_string[++ii]=strtok(NULL, " ")) != NULL)
   {
       printf("%s\n", sub_string[ii]);
   }
}

Basically, strtok inserts '\0' at the delimiter found, and returns a pointer to the start of the token. So you don't actually need to allocate new memory to the sub_string elements - just to the array. I set the number of elements of the array to 50; in reality you will want to make sure that your while loop stops before you run out of space…

A little diagram might help:

Original string:

s t r i n g    t o    s p l i t    u p \0

After first call to strtok:

s t r i n g \0 t o    s p l i t    u p \0

^ first pointer

After next call:

s t r i n g \0 t o \0 s p l i t    u p \0

               ^ second pointer

After the third call:

s t r i n g \0 t o \0 s p l i t \0 u p \0

                      ^ third pointer

etc.

If you want to store the substrings in "different variables" (not sure why you don't consider sub_string[0] and sub_string[1] etc to be convenient 'different variables', but I'll leave that for another time), you could change the above to:

#include <stdio.h>
#include <string.h>

int main(void)
{

   char test_string[50]="string to split up";
   char *string1, *string2, *string3, *string4, *string5;
   int n = 0;
   /* Extract first string */
   if(strlen(test_string)==0) return 0;

   string1=strtok(test_string, " "));
   n++;
   printf("%s\n", string1);

   while( n < 5 ) {
     string2 = strtok(NULL, " ");
     if (string2 == NULL) break; else n++;
     string3 = strtok(NULL, " ");
     if (string3 == NULL) break; else n++;
     string4 = strtok(NULL, " ");
     if (string4 == NULL) break; else n++;
     string5 = strtok(NULL, " ");
     if (string4 == NULL) break; else n++;
  }
  printf("the total number of strings found is %d\n", n);
}

I just don't think it is nearly as elegant as using an array of char *. Can you see my point?

Answer 2

#include <stdio.h>
#include <string.h>

main()
{

   char test_string[50]="string to split up";
   char *sub_string;
   char withoutspace[50];

   /* Extract first string */
   printf("%s\n", strtok(test_string, " "));

   /* Extract remaining strings   */
   while ( (sub_string=strtok(NULL, " ")) != NULL)
   {
       strcat(withoutspace, sub_string); //printf("%s\n", sub_string);
   }
}