Re: [PERFORM] A Better External Sort?

On 10/6/05, Michael Stone <mstone+postgres@mathom.us> wrote:
> On Wed, Oct 05, 2005 at 11:24:07AM -0400, Luke Lonergan wrote:
> >Nope - it would be disk wait.
>
> I said I/O overhead; i.e., it could be the overhead of calling the
> kernel for I/O's. E.g., the following process is having I/O problems:
>
> time dd if=/dev/sdc of=/dev/null bs=1 count=10000000
> 10000000+0 records in
> 10000000+0 records out
> 10000000 bytes transferred in 8.887845 seconds (1125132 bytes/sec)
>
> real    0m8.889s
> user    0m0.877s
> sys     0m8.010s
>
> it's not in disk wait state (in fact the whole read was cached) but it's
> only getting 1MB/s.
>
> Mike Stone
>
> ---------------------------(end of broadcast)---------------------------
> TIP 5: don't forget to increase your free space map settings
>
I think you only proved that dd isn't the smartest tool out there... or
that using it with a blocksize of 1 byte doesn't make too much sense.


[andrej@diggn:~]$ time dd if=/dev/sr0 of=/dev/null bs=2048 count=4883
4883+0 records in
4883+0 records out

real    0m6.824s
user    0m0.010s
sys     0m0.060s
[andrej@diggn:~]$ time dd if=/dev/sr0 of=/dev/null bs=1 count=10000000
10000000+0 records in
10000000+0 records out

real    0m18.523s
user    0m7.410s
sys     0m10.310s
[andrej@diggn:~]$ time dd if=/dev/sr0 of=/dev/null bs=8192 count=1220
1220+0 records in
1220+0 records out

real    0m6.796s
user    0m0.000s
sys     0m0.070s

That's with caching, and all.  Or did I miss the point of your post
completely?  Interestingly, the CPU usage with the bs=1 goes up
to 97%, it stays at a mellow 3% with the 8192 and 2048.


Cheers,
Andrej