Re: ANALYZE sampling is too good

On 10/12/13 15:04, Mark Kirkwood wrote:
> On 10/12/13 13:53, Mark Kirkwood wrote:
>> On 10/12/13 13:20, Mark Kirkwood wrote:
>>> On 10/12/13 13:14, Mark Kirkwood wrote:
>>>> On 10/12/13 12:14, Heikki Linnakangas wrote:
>>>>>
>>>>>
>>>>> I took a stab at using posix_fadvise() in ANALYZE. It turned out 
>>>>> to be very easy, patch attached. Your mileage may vary, but I'm 
>>>>> seeing a nice gain from this on my laptop. Taking a 30000 page 
>>>>> sample of a table with 717717 pages (ie. slightly larger than 
>>>>> RAM), ANALYZE takes about 6 seconds without the patch, and less 
>>>>> than a second with the patch, with effective_io_concurrency=10. If 
>>>>> anyone with a good test data set loaded would like to test this 
>>>>> and post some numbers, that would be great.
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> I did a test run:
>>>>
>>>> pgbench scale 2000 (pgbench_accounts approx 25GB).
>>>> postgres 9.4
>>>>
>>>> i7 3.5Ghz Cpu
>>>> 16GB Ram
>>>> 500 GB Velociraptor 10K
>>>>
>>>> (cold os and pg cache both runs)
>>>> Without patch:  ANALYZE pgbench_accounts    90s
>>>> With patch: ANALYZE pgbench_accounts  91s
>>>>
>>>> So I'm essentially seeing no difference :-(
>>>
>>>
>>> Arrg - sorry forgot the important bits:
>>>
>>> Ubuntu 13.10 (kernel 3.11.0-14)
>>> filesystem is ext4
>>>
>>>
>>>
>>
>> Doing the same test as above, but on a 80GB Intel 520 (ext4 
>> filesystem mounted with discard):
>>
>> (cold os and pg cache both runs)
>> Without patch:  ANALYZE pgbench_accounts  5s
>> With patch: ANALYZE pgbench_accounts  5s
>>
>>
>>
>>
>>
>
> Redoing the filesystem on the 520 as btrfs didn't seem to make any 
> difference either:
>
> (cold os and pg cache both runs)
> Without patch:  ANALYZE pgbench_accounts  6.4s
> With patch: ANALYZE pgbench_accounts  6.4s
>
>
>

Ah - I have just realized I was not setting effective_io_concurrency - 
so I'll redo the test. - Apologies.

Regards

Mark