C string.h strtok() Function

strtok() Function

The strtok() function in C is used to break a string into a sequence of tokens based on specified delimiter characters. It processes the input string in a series of calls, returning one token per call until the entire string has been tokenized.

---

Syntax of strtok()

char *strtok(char *str, const char *delimiters);

Parameters

Parameter	Description
str	C string to be tokenized. This string is modified by inserting null characters to terminate tokens.
delimiters	C string containing the set of delimiter characters that separate tokens.

On the first call, the function requires a non-null string to start tokenization. For subsequent calls to continue tokenizing the same string, a null pointer must be provided as the first argument. The function scans the string for the first character not in the set of delimiters, marking the start of a token, and then searches for the next delimiter to mark the end. It replaces the delimiter with a null-character to isolate the token, and maintains an internal state to allow successive calls to continue from where it left off.

Return Value

If a token is found, strtok() returns a pointer to the beginning of the token; otherwise, it returns a null pointer when no more tokens are available.

---

Examples for strtok()

Example 1: Basic Tokenization

This example demonstrates how to tokenize a string using a single delimiter.

Program

#include <stdio.h>
#include <string.h>

int main() {
    char str[] = "Hello,World,How,Are,You";
    char *token = strtok(str, ",");
    
    while (token != NULL) {
        printf("%s\n", token);
        token = strtok(NULL, ",");
    }
    return 0;
}

Explanation:

1. A character array str is initialized with a comma-separated string.
2. The first call to strtok() tokenizes the string using the comma as the delimiter.
3. A loop is used to repeatedly call strtok() with a null pointer to continue tokenizing the same string.
4. Each token is printed on a new line.

Output:

Hello
World
How
Are
You

Example 2: Tokenization with Multiple Delimiters

This example demonstrates how to tokenize a string using more than one delimiter character.

Program

#include <stdio.h>
#include <string.h>

int main() {
    char str[] = "apple,banana;cherry orange";
    char *token = strtok(str, ",; ");
    
    while (token != NULL) {
        printf("%s\n", token);
        token = strtok(NULL, ",; ");
    }
    return 0;
}

Explanation:

1. A character array str contains a string with words separated by commas, semicolons, and spaces.
2. The first call to strtok() uses the delimiters ,; to split the string.
3. A loop continues to extract each token until no more tokens are available.
4. Each token is printed on a separate line.

Output:

apple
banana
cherry
orange