Re: [psycopg] speed concerns with executemany()

On Mon, Jan 2, 2017 at 5:16 PM, Karsten Hilbert <Karsten.Hilbert@gmx.net> wrote:
> On Mon, Jan 02, 2017 at 05:10:10PM +0100, Daniele Varrazzo wrote:
>
>> > Any chance 'l' can be supported to be a generator expression ?
>>
>> The function would consume only page_size records at time. This is to
>> avoid creating a humongous query in memory and then on the server, but
>> has the nice effect of requesting only as many records from the
>> sequence.
>
> Nice.
>
> Can said sequence be a generator (IOW not needing to support
> len() upfront) ?

Correct: see the implementation of paginate() in the gist at
<https://gist.github.com/dvarrazzo/4204cca5d1cd7c9c95bc814d81e6b83e>:
the sequence is only consumed via next(). Final implementation may
change but accessing the sequence only as a generator is a desired
feature for me too.

-- Daniele