Re: new server I/O setup

On Thu, Jan 14, 2010 at 1:03 PM, Fernando Hevia <fhevia@ip-tel.com.ar> wrote:
> Hi all,
>
> I've just received this new server:
> 1 x XEON 5520 Quad Core w/ HT
> 8 GB RAM 1066 MHz
> 16 x SATA II Seagate Barracuda 7200.12
> 3ware 9650SE w/ 256MB BBU
>
> It will run an Ubuntu 8.04 LTS Postgres 8.4 dedicated server. Its database
> will be getting between 100 and 1000 inserts per second (those are call
> detail records of ~300 bytes each) of around 20 clients (voip gateways).
> Other activity is mostly read-only and some non time-critical writes
> generally at off peak hours.
>
> So my first choice was:
>
> 2 discs in RAID 1 for OS + pg_xlog partitioned with ext2.
> 12 discs in RAID 10 for postgres data, sole partition with ext3.
> 2 spares
>
>
> My second choice is:
>
> 4 discs in RAID 10 for OS + pg_xlog partitioned with ext2
> 10 discs in RAID 10 for postgres, ext3
> 2 spares.
>
> The bbu caché will be enabled for both raid volumes.
>
> I justified my first choice in that WAL writes are sequentially and OS
> pretty much are too, so a RAID 1 probably would hold ground against a 12
> disc RAID 10 with random writes.

I think your first choice is right.  I use the same basic setup with
147G 15k5 SAS seagate drives and the pg_xlog / OS partition is almost
never close to the same level of utilization, according to iostat, as
the main 12 disk RAID-10 array is.  We may have to buy a 16 disk array
to keep up with load, and it would be all main data storage, and our
pg_xlog main drive pair would be just fine.

> I don't know in advance if I will manage to gather enough time to try out
> both setups so I wanted to know what you guys think of these 2
> alternatives. Do you think a single RAID 1 will become a bottleneck? Feel
> free to suggest a better setup I hadn't considered, it would be most
> welcome.

For 12 disks, most likely not.  Especially since your load is mostly
small randomish writes, not a bunch of big multi-megabyte records or
anything, so the random access performance on the 12 disk RAID-10
should be your limiting factor.

> Pd: any clue if hdparm works to deactive the disks write cache even if they
> are behind the 3ware controller?

Not sure, but I'm pretty sure the 3ware card already does the right
thing and turns off the write caching.