# Split a string by a delimiter

Tokenizing a string in C typically involves breaking a string into smaller pieces or tokens based on a specified delimiter. The standard library provides a function called `strtok` that can be used for this purpose. Here's a simple example with comments explaining each step:

```c
#include <stdio.h>
#include <string.h>

int main() {
    // Input string to tokenize
    char inputString[] = "Hello,World,Tokenization,Example";

    // Delimiter used for tokenization (comma in this case)
    const char delimiter[] = ",";

    // Token pointer to store the current token during iteration
    char *token;

    // Use strtok to get the first token
    token = strtok(inputString, delimiter);

    // Continue getting tokens until strtok returns NULL
    while (token != NULL) {
        // Print the current token
        printf("Token: %s\n", token);

        // Use strtok with NULL to continue tokenization
        token = strtok(NULL, delimiter);
    }

    return 0;
}
```

In the provided C code, a string is tokenized using the `strtok` function, which breaks the input string, "Hello,World,Tokenization,Example," into smaller pieces based on the specified delimiter (`,`). The program iterates through the resulting tokens, printing each one on a new line. The tokenization process begins by calling `strtok` with the input string and delimiter, and subsequent calls use `NULL` as the first argument to continue tokenization. The token pointer is updated during each iteration, and the loop continues until there are no more tokens. This example demonstrates a basic tokenization scenario, but it's important to note that `strtok` modifies the original string, and complex cases may require additional handling and error checks.