Re: OutOfMemory hibernate scroll with 2M records | Postgresql 8.4 DB

On Mon, Dec 21, 2009 at 12:54 PM, Ankit Kumar <ankitk@xebia.com> wrote:
> I am running a Criteria.scroll() on postgresql on a DB containing 2M
> records. The memory keeps increasing and finally it generates an
> OutOfMemoryException. Please can you advice how to fix this.
>
> Postgresql DB version: 8.4
> Postgresql Driver Used: postgresql-8.4-701.jdbc4.jar
>
> Some forums do mention that "scroll" is slow on Postgresql.
>
> Is this a known issue or we need to do something specific to make scroll
> work on Postgresql?
>
>
> Exception StackTrace:
> java.lang.OutOfMemoryError: Java heap space
>

This looks like a Java error, not a Postgres one. It looks like
Hibernate is asking for the entire result set back from Postgres
instead of using a scrollable cursor and is trying to hold it all in
RAM. You should ask the Hibernate people if they plan to either spill
to disk or use database-side cursors which certainly do if necessary.

--
greg