TPC (was Great Bridge benchmark results for Postgres, 4 others)

A more interesting benchmark would be to compare TPC/C results on same
kind of hardware other vendors use for THEIR TPC benchmarks, which are
posted on tpc.org, as well as comparing price/performance of each.

TPC as run by company 'commissioned by GB' cannot be validated and
accepted into TPC database, they must be run under close supervision by
TPC-approved monitors. I hope GB actually springs for the price of running
the REAL TPC benchmark (last I heard it was around 25k$).

To see how postgres performs on low-end (for TPC low-end is <8 processors)
would be interesting to say the least.

A problem with a real TPC is the strong suggestion to run a transaction
manager, to improve speed. No transaction manager supports postgres yet.

Another note on TPC is that they require to include as a final price
support contract, on which GreatBridge should be able to compete.


On Mon, 14 Aug 2000, Ned Lilly wrote:

> Marc's right... we opted for ODBC to ensure as much of an "apples to apples"
> comparison as possible.  Of the 5 databases we tested, a native driver existed for
> only the two (ahem) unnamed proprietary products - Postgres, Interbase, and MySQL
> had to rely on ODBC.  So we used the vendor's own ODBC for each of the other two
> cases.
>
> <disclaimer>
> As with all benchmarks, your mileage will vary according to hardware, OS, and of
> course the specific application.  What we attempted to do here was use two
> industry-standard benchmarks and treat all five products the same.
> </disclaimer>
>
> Presumably, if the vendor had taken the time to write a native driver for
> Postgres, the results would have seen an even bigger kick.  We don't have any
> reason to think that the results for all five tests in native driver mode would be
> out of proportion to the results we got through ODBC.
>
> Regards,
> Ned
>
>
> The Hermit Hacker wrote:
>
> > On Mon, 14 Aug 2000, Steve Wolfe wrote:
> >
> > > > 1) Using only ODBC drivers.  I don't know how much of an impact a driver
> > > can
> > > > make but it would seem that using native drivers would shutdown one source
> > > > of objections.
> > >
> > >   Using ODBC is guaranteed to slow down the benchmark.  I've seen native
> > > database drivers beat ODBC by anywhere from a factor of two to an order of
> > > magnitude.
> >
> > I haven't had a chance to take a look at the benchmarks yet, having just
> > seen this, but *if* Great Bridge performed their benchmarks such that all
> > the databases were access via ODBC, then they are using an
> > 'apples-to-apples' approach, as each will have similar slowdowns as a
> > result ...
>
>