Re: [HACKERS] Re: [GSOC 17] Eliminate O(N^2) scaling fromrw-conflict tracking in serializable transactions

Hi,  Alvaro and Kevin.

> Anyway, this is just my analysis. 
> So I want to hack the PG and count the conflict lists' size of transactions. That would be more accurate.

In the last week, I hacked the PG to add an additional thread to count RWConflict list lengths. 
And tune the benchmark to get more conflict. But the result is still not good.

> 
> > 
> > Yeah, you need a workload that generates a longer conflict list -- if
> > you can make the tool generate a conflict list with a configurable
> > length, that's even better (say, 100 conflicts vs. 1000 conflicts).
> > Then we can see how the conflict list processing scales.
> > 
> 
> Yes, I tried to increase the read set to make more conflicts. 
> However the abort ratio will also increase. The CPU cycles consumed by conflict tracking are still less than 1%.
> According to the design of PG, a transaction will be aborted if there is a rw-antidependency. 
> In this case, a transaction with a longer conflict list, is more possible to be aborted.
> That means, the conflict list doesn't have too many chances to grow too long. 
> 

To solve this problem, I use just two kinds of transactions: Read-only transactions and Update-only transactions.
In this case, no transaction would  have an In-RWconflict and an Out-RWconflict at the same time.  
Thus transactions would not be aborted by conflict checking. 

Specifically, The benchmark is like this:
The table has 10K rows. 
Read-only transactions read 1K rows and Update-only transactions update 20 random rows of the table. 

In this benchmark, about 91% lists are shorter than 10; 
lengths of 6% conflict lists are between 10 and 20. Only 2% lists are longer than 20. The CPU utilization of
CheckForSerializableConflictOut/Inis 0.71%/0.69%.
 

I tried to increase the write set. As a result, conflict list become longer. But the total CPU utilization is decreased
(about50%).
 
CPU is not the bottleneck anymore. I'm not familiar with other part of PG. Is it caused by LOCK? Is there any chance to
getrid of this problem?
 

BTW, I found that the email is not very convenient, especially when I  have some problem and want to discuss with you.
Would you mind scheduling a meeting every week by Skye, or other instant message software you like?
I would not take you too much time. Maybe one hour is enough.   


Sincerely.
--
Mengxing Liu