Re: [HACKERS] follow-up on PC Week Labs benchmark results

At 02:54 PM 2/7/00 +0000, Thomas Lockhart wrote:

>> ... PC Week never benchmarks non-production, non-released code...
>> ... this story was driven by the news of
>> InterBase going open source.
>
>Hmm. InterBase going open source seems to be a pre-alpha vaporware
>feature so far ;)

This is a good point as InterBase announced that their upcoming
Beta was going open source.

Did you test an early copy of their upcoming beta, or did you test
their current, non-Open Source product?

If you tested their upcoming beta, then it would seem fair to test
our upcoming beta, too :)  If you tested their non-Open Source
current production version, then you're not testing two Open Source
databases...hmmm...

Of course, InterBase may've expanded on their earlier announcement
of what's going Open Source, I've not been tracking it.

>> In particular, I'd like to compare IB 6 with PG 7.x and MySQL later this
>> year (I didn't benchmark MySQL this time because I ran out of time, but
>> would very much like to.)

Actually, this slipped by me the first time.

Why benchmark MySQL?  It's not a real RDBMS, it doesn't even pretend
to support ACID semantics.  Clearly it is going to be faster than 
databases that do because supporting ACID semantics is expensive.

This would be comparing apples with oranges, meaningless.

Now, don't get me wrong, for many application spaces mySQL is fine.  If
you're running a bboard system for overclockers, for instance, you probably
would sigh in relief if disaster struck and you lost all your data.

On the other hand, if you're running an e-commerce site losing data is
not cool and mySQL is not appropriate.

Rather than benchmark, it would seem more useful to educate your readers
about the meaning of ACID, and how to decide when you need it and when you
don't.  That would seem far more important, because in my experience many
people don't understand that there is a real difference between a program
that executes a subset of SQL in a simple manner, and an RDBMS that 
passes the ACID test and happens to be driven by SQL queries.

If you were to benchmark in the context of such an article, it would make
some sense, because you could do so in order to answer the question, "How
much does ACID hurt performance?"  

This would give your readers real information to help drive their choice
of software. 



- Don Baccus, Portland OR <dhogaza@pacifier.com> Nature photos, on-line guides, Pacific Northwest Rare Bird Alert
Serviceand other goodies at http://donb.photo.net.