Re: pgbench results arent accurate

If you have not amended any Postgres config parameters, then you'll get 
checkpoints approx every 5 min or so. Thus using a Pgbench run time of 
5min is going sometimes miss/sometimes hit a checkpoint in progress - 
which will hugely impact test results.

I tend to do Pgbench runs of about 2x checkpoint_timeout - (i.e 10 min 
for default configurations). Also for increased repeatability, I do a 
manually triggered checkpoint immediately before each run.

regards

Mark

On 13/12/18 1:53 AM, Mariel Cherkassky wrote:
> Hey,
> I installed a new postgres 9.6 on both of my machines. I'm trying to 
> measure the differences between the performances in each machine but 
> it seems that the results arent accurate.
> I did 2 tests :
>
> 1)In the first test the scale was set to 100 :
> pgbench -i -s 100 -U postgres -d bench -h machine_name
> pgbench -U postgres -d bench  -h machine_name -j 2 -c 16 -T 300
> RUN        TPS - machine1     TPS-machine2
> 1     697     555
> 2     732     861
> 3     784     842
>
>     
>     
>
>
> 2)In this test the scale was set to 10000 :
> pgbench -i -s 10000 -U postgres -d bench -h machine_name
> pgbench -U postgres -d bench --progress=30 -h machine_name -j 2 -c 16 
> -T 300
> RUN     TPS-MACHINE1     TPS-MACHINE2
> 1     103     60
> 2     63     66
> 3     74     83
> 4     56     61
> 5     75     53
> 6     73     60
> 7     62     53
>
>
> In both cases after the initalization I restarted the database and 
> cleared the cashe(echo 1 > /proc/sys/vm/drop_caches) one time. During 
> all the runs I didnt shutdown the machine.
>
> Now, I was hopping the the tps will be almost the same in each machine 
> for all the runs. In other words, I wanted to see that the tps in 
> machine1 during all the tps are almost the same but I see that the 
> values arent accurate.
>
> Any idea what might cause the differences in every run ?