[Discuss] 'C' string tokenizer for those who hate strtok

[Adam Parkin]

David Bronaugh wrote:
> What's so hard about @foo = split(/\//, $str)?

Well aside from the fact I can't figure out why you use that particular 
regex, nothing. =;->  (I thought it was supposed to split a line on 
whitespace in which case \s+ seems the more typical approach)

> The point is that you're comparing apples and oranges. Perl and Python 
> are both significantly slower at runtime and significantly faster to write.

Are you telling me you can see a huge difference in the runtimes of the 
C version versus the Perl/Python version?

I'm not disagreeing with you that Perl/Python has a cost associated with 
it, what I'm saying is that in an awful lot of problem domains the 
difference is so minor (particularly on modern machines) that it becomes 
difficult to see why one would choose the "fast" compiled language 
rather than the "slow" interpreted one.

Sure you can always come up with the specific case where program X 
written in C runs 50% faster than program Y written in Perl, but in the 
general case it's often hard to see the performance benefits of C over 
other languages, but easy to see the maintenance, safety, and SENG 
benefits of other languages over C.

> Write Ocaml. Faster than C to write, and runs at around the same speed 
> as C. Oh, and it actually has error checking.

First thing: please point me in the direction of some hard data that 
shows that C is so much faster than Perl/Python, and that OCaml is so 
much closer to C in some real-world problems.  (this isn't a sarcastic 
comment, I genuinely would like to see this)

Next thing: well, while I haven't used Ocaml, I have written code in SML 
and it is almost always the case that it takes far longer to write 
something in ML (although it is much fewer LOC, the amount of "thinking 
about the problem" increases dramatically).  Don't get me wrong, I think 
functional languages will play a larger role in big software projects in 
the future, I just have difficulty with the premise that because 
functional code is shorter it must be faster to write (perhaps I'm 
misunderstanding what you intended here though -- straw man at work).

I've been following this thread pretty closely and what really surprises 
me is that nobody seems to realize that some problems simply lend 
themselves to some languages/language paradigms.  Writing a simluation 
of the interaction between real-world entities?  Makes sense to model 
those entities as object in an OO language.  Writing Matlab?  Makes 
sense to implement it in a functional language.  That doesn't mean you 
can't do Matlab in an OO language, or the simulation in a functional 
one, it just means you're going to be making the language fit the 
problem rather than making the problem fit the language.

So to say that you only use language X for code that has to be "safe" or 
language Y for code that has to be "fast" is ludicrous IMHO.  You choose 
the language that most easily allows you to express a solution to your 
problem while falling within the constraints of memory/runtime for said 
problem.  To do otherwise is to fall back on the dogmatic BS that caused 
people to stick with goto for far too long. (hey you can't have a debate 
on programming languages without bringing up goto at some point...) =8-p
---
Adam Parkin
E-mail: pzelnip at telus.net
-----------------------