Re: libpq single-row mode performance

Hello, James.

You wrote:

JD> Hi,
JD>  
JD> I’m writing an app on top of libpq. To avoid running out of
JD> memory I’m using the single-row mode, but am finding the overhead in using this quite significant.
JD>  
JD> What I see is that ~60% of my application’s run time when
JD> retrieving data is spent in calls to either PQgetResult() and
JD> PQclear(). I’ve added a switch in my app to turn off single-row
JD> mode and the performance roughly doubles.
JD>  
JD> Would it be possible to optimize the single-row mode? For
JD> example, add an API call PQnextRow(PGConn*), which is only usable
JD> in single-row mode and will just update the existing result with
JD> the contents of the next row? This would let us avoid to overhead of:
JD> 1.       Malloc’ing a PGResult and initializing its defaults.
JD> 2.       Copying the column metadata to the new results.
JD> 3.       Possibly we can avoid malloc’ing cell data if the next
JD> row has cells the same size or smaller than a previous row, though
JD> I’m not sure of the internals here.
JD> 4.       Release memory with PQclear().
JD>  
JD> James Duong | Senior Computer Scientist | Simba Technologies Inc.
JD> Tel +1.604.633.0008 ext. 120 | Fax +1.604.633.0004 | jamesd@simba.com
JD>  
JD> 938 West 8th Avenue | Vancouver, BC | Canada | V5Z 1E5
JD> The Data Access and Analytics Experts  |  www.simba.com 
JD>  
JD>                 
JD>  
JD> This email message is for the sole use of the intended
JD> recipient(s) and may contain confidential and privileged
JD> information.  Any unauthorized review, use, disclosure, or
JD> distribution is prohibited.  If you are not the intended
JD> recipient, please contact the sender by reply email and destroy
JD> all copies of the original message.  Thank you.
JD>  

Good idea. +1 from me. Should we send this message to pg-hackers as
well?

-- 
With best wishes,Pavel                          mailto:pavel@gf.microolap.com