Re: Relation extension scalability

Поиск
Список
Период
Сортировка
От Dilip Kumar
Тема Re: Relation extension scalability
Дата
Msg-id CAFiTN-vwU9=1r=6Vc7+H8+2oZpfO-1bWMrYhsh9CDQd86jD_Dw@mail.gmail.com
обсуждение исходный текст
Ответ на Re: Relation extension scalability  (Amit Kapila <amit.kapila16@gmail.com>)
Ответы Re: Relation extension scalability  (Dilip Kumar <dilipbalaut@gmail.com>)
Список pgsql-hackers

On Tue, Mar 1, 2016 at 4:36 PM, Amit Kapila <amit.kapila16@gmail.com> wrote:
One thing that is slightly unclear is that whether there is any overhead due to buffer eviction especially when the buffer to be evicted is already dirty and needs XLogFlush().  One reason why it might not hurt is that by the time we tries to evict the buffer, corresponding WAL is already flushed by WALWriter or other possibility could be that even if it is getting done during buffer eviction, the impact for same is much lesser.  Can we try to measure the number of flush calls which happen during buffer eviction?


Good Idea, I will do this test and post the results..
 
 
Proc Sleep test for Insert test when data don't fits in shared buffer and inserting big record of 1024 bytes, is currently running
once I get the data will post the same.


Okay.  However, I wonder if the performance data for the case when data doesn't fit into shared buffer also shows similar trend, then it might be worth to try by doing extend w.r.t load in system.  I mean to say we can batch the extension requests (as we have done in ProcArrayGroupClearXid) and extend accordingly, if that works out then the benefit could be that we don't need any configurable knob for the same.

1. One option can be as you suggested like ProcArrayGroupClearXid, With some modification, because when we wait for the request and extend w.r.t that, may be again we face the Context Switch problem, So may be we can extend in some multiple of the Request.
(But we need to take care whether to give block directly to requester or add it into FSM or do both i.e. give at-least one block to requester and put some multiple in FSM)


2. Other option can be that we analyse the current Load on the extend and then speed up or slow down the extending speed.
Simple algorithm can look like this

If (GotBlockFromFSM())
  Success++                  // We got the block from FSM
Else
  Failure++                    // Did not get Block from FSM and need to extend by my self

Now while extending
-------------------------
Speed up
-------------
If (failure - success > Threshold )  // Threshold can be one number assume 10.
    ExtendByBlock += failure- success;       --> If many failure means load is high then Increase the ExtendByBlock
    Failure = success= 0                             --> reset after this so that we can measure the latest trend.

Slow down..
--------------
//Now its possible that demand of block is reduced but ExtendByBlock is still high.. So analyse statistic and slow down the extending pace..

If (success - failure >  Threshold)
{  
// Can not reduce it by big number because, may be more request are satisfying because this is correct amount, so gradually decrease the pace and re-analyse the statistics next time.
   ExtendByBlock --;                   
   Failure = success= 0
}

Any Suggestions are Welcome...


--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com

В списке pgsql-hackers по дате отправления:

Предыдущее
От: Michael Paquier
Дата:
Сообщение: Re: TAP / recovery-test fs-level backups, psql enhancements etc
Следующее
От: Pavel Stehule
Дата:
Сообщение: Re: proposal: PL/Pythonu - function ereport