[Discuss] 'C' string tokenizer for those who hate strtok

[Paul Nienaber]

p.willis at telus.net wrote:
> Quoting p.willis at telus.net:
>
>   
>> Quoting Paul Nienaber <phox at phox.ca>:
>>
>>     
>>> Buffer from nowhere?  POSIX has mandated strtok_r() for like... ever. 
>>> strtok() _is_ stupid.  It's also about one more line to use strchr() or
>>> one can use BSD strsep(), or whatever...
>>>
>>> ~p
>>>       
>> Paul,
>>
>> It's fluff. It's a learning excercise for 'C' linked lists for beginners.
>>
>> It's entertainment...or would you rather read about partitioning hard drives
>> 200 more times.
>>
>> Peter
>>     
>
>
> I should also note that this technique is better than all 
> of the above mentioned tokenization routines in that it doesn't
> destroy the original data. It always makes me wonder about
> libraries when the actual 'man pages' say to avoid the routine
> if possible.
> (ie: strtok, strtok_r, and strsep all come with this warning)
>
> A second point regarding the storage is that the deallocation
> is obviated by this technique reducing memory leaks. But that's
> splitting hairs since free() also works [most of the time] for
> some of the other routines.
>   
Yeah.  It's way better to allocate another buffer for every token,
rather than copying the whole thing and delimiting it, which of course
makes your technique "not better", because it incurs a whole pile more
calls to malloc() and friends...  </rant>  (but you were the one who
decided to use the word "better"...)

I should come clean here and mention that I've taught C at UVic on at
least one occasion.  I won't even go into the C-specific issues here.

~p