On Sep 21, 2014, at 12:35 AM, Craig Ringer <craig@2ndquadrant.com> wrote:
> On 09/21/2014 11:24 AM, Lussier, Denis wrote:
>> This does seem very worthwhile. Can someone please sketch out a
>> mini-design and see if it makes sense to the pgjdbc core? I'd be
>> willing to hack some code, but, I'd want the design to be pre-vetted.
>
> Actually, on second thought, if we're going to do this we'd be silly to
> restrict it to spilling to disk.
>
> What we should have is the ability to flush a resultset to non-memory
> storage that provides a given interface when it exceeds a given size.
This all sounds very interesting, but should it really be baked into the driver?
Shouldn’t such a mechanism be built on top of the JDBC API? Then it could work
independently of the Driver implementation.
Additionally, if this does get implemented, please leave it off by default. We
have many SSDs backing our database server and very little space / IOPS on
application nodes (intentionally, and I’m not sure we are the only ones) so
suddenly spilling to disk could be disastrous for our performance.