[Discuss] 'C' string tokenizer for those who hate strtok

[Brian Quinlan]

David Bronaugh wrote:
> For amusement I rewrote your main.c file... it was amusing.

For fun, I rewrote it in Python. Note that this would be considered bad 
Python code, but I tried to follow the same call pattern as the C code:

def Tokenize(tokens, string, delimiter):
     tokens.extend((token for token in string.split(delimiter)
                    if len(token) > 0))
     return len(tokens)

tokens = [] #
print Tokenize(tokens, "This/is/a/delimted/string/", '/')
print tokens
print Tokenize(tokens, "apples, oranges, grapes", ',')
print tokens

Output:
5
['This', 'is', 'a', 'delimted', 'string']
8
['This', 'is', 'a', 'delimted', 'string', 'apples', ' oranges',
  ' grapes']


Cheers,
Brian