Re: Speed / Server

On Sun, 4 Oct 2009, anthony@resolution.com wrote:

> The nasty part of this problem is that the data needs to be "readily"
> available for reports, and we cannot consolidate the data for reporting
> purposes.

Just because you have to store the detailed data doesn't mean you can't
store a conslidated view on it too.  Have you considered driving the
primary reporting off of materialized views, so you only compute those
once?

> I know we need a LOT of RAM (as much as we can afford), and we're looking
> at a couple of Nehalem systems w/ a large, and fast, RAID-10 disk set up.

There is a lot of variation in RAID-10 setups that depends on the
controller used.  Make sure you're careful to consider the controller card
and performance of its battery-backed cache a critical component here;
performance does not scale well with additional drives if your controller
isn't good.

What card are you using now for your RAID-1 implementation?

> 1.  Other than partitioning (by system, and/or date), and splitting up the
> data into multiple tables (by data type), what could be done within
> Postgresql to help with this type of set up (1 large table)?

This seems like a perfect fit for partitioning by date.

> 2.  Before going out and buying a beast of a system, we'd like to get some
> idea of performance on a "high-end" system.  We may need to split this up,
> or move into some other type of architecture.  Do you know of anyone who
> would let us "play" with a couple of systems to see what would be an
> applicable purchase?

Find vendors who sell things you like and ask if they have an eval system
available.  As prices move up, those become more common.

--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD