OutOfMemory hibernate scroll with 2M records | Postgresql 8.4 DB
| От | Ankit Kumar |
|---|---|
| Тема | OutOfMemory hibernate scroll with 2M records | Postgresql 8.4 DB |
| Дата | |
| Msg-id | 4B3047E7.9020901@xebia.com обсуждение исходный текст |
| Ответы |
Re: OutOfMemory hibernate scroll with 2M records | Postgresql
8.4 DB
Re: OutOfMemory hibernate scroll with 2M records | Postgresql 8.4 DB Re: OutOfMemory hibernate scroll with 2M records | Postgresql 8.4 DB Re: OutOfMemory hibernate scroll with 2M records | Postgresql 8.4 DB |
| Список | pgsql-jdbc |
Hi
I am running a Criteria.scroll() on postgresql on a DB containing 2M records. The memory keeps increasing and finally it generates an OutOfMemoryException. Please can you advice how to fix this.
Postgresql DB version: 8.4
Postgresql Driver Used: postgresql-8.4-701.jdbc4.jar
Some forums do mention that "scroll" is slow on Postgresql.
Is this a known issue or we need to do something specific to make scroll work on Postgresql?
It appears the driver class is loading all entity objects in memory and not releasing them as expected in a scroll operation.
Appreciate any guidance/suggestion.
Exception StackTrace:
java.lang.OutOfMemoryError: Java heap space
at java.lang.Class.getDeclaredFields0(Native Method)
at java.lang.Class.privateGetDeclaredFields(Class.java:2291)
at java.lang.Class.getDeclaredField(Class.java:1880)
at java.util.concurrent.atomic.AtomicReferenceFieldUpdater$AtomicReferenceFieldUpdaterImpl.<init>(AtomicReferenceFieldUpdater.java:181)
at java.util.concurrent.atomic.AtomicReferenceFieldUpdater.newUpdater(AtomicReferenceFieldUpdater.java:65)
at java.sql.SQLException.<clinit>(SQLException.java:353)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1777)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257)
at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:479)
at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:367)
at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271)
at org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:93)
at org.hibernate.jdbc.AbstractBatcher.getResultSet(AbstractBatcher.java:186)
at org.hibernate.loader.Loader.getResultSet(Loader.java:1787)
at org.hibernate.loader.Loader.scroll(Loader.java:2286)
at org.hibernate.loader.criteria.CriteriaLoader.scroll(CriteriaLoader.java:89)
at org.hibernate.impl.SessionImpl.scroll(SessionImpl.java:1533)
at org.hibernate.impl.CriteriaImpl.scroll(CriteriaImpl.java:297)
at org.hibernate.impl.CriteriaImpl.scroll(CriteriaImpl.java:291)
at com.hi.openname.dao.impl.HibernateKnowledgeItemDao.loadAllCommitted(HibernateKnowledgeItemDao.java:104)
at com.hi.openname.dao.impl.HibernateKnowledgeItemDao.loadAllCommitted(HibernateKnowledgeItemDao.java:1)
Regards
Ankit
I am running a Criteria.scroll() on postgresql on a DB containing 2M records. The memory keeps increasing and finally it generates an OutOfMemoryException. Please can you advice how to fix this.
Postgresql DB version: 8.4
Postgresql Driver Used: postgresql-8.4-701.jdbc4.jar
Some forums do mention that "scroll" is slow on Postgresql.
Is this a known issue or we need to do something specific to make scroll work on Postgresql?
It appears the driver class is loading all entity objects in memory and not releasing them as expected in a scroll operation.
Appreciate any guidance/suggestion.
Exception StackTrace:
java.lang.OutOfMemoryError: Java heap space
at java.lang.Class.getDeclaredFields0(Native Method)
at java.lang.Class.privateGetDeclaredFields(Class.java:2291)
at java.lang.Class.getDeclaredField(Class.java:1880)
at java.util.concurrent.atomic.AtomicReferenceFieldUpdater$AtomicReferenceFieldUpdaterImpl.<init>(AtomicReferenceFieldUpdater.java:181)
at java.util.concurrent.atomic.AtomicReferenceFieldUpdater.newUpdater(AtomicReferenceFieldUpdater.java:65)
at java.sql.SQLException.<clinit>(SQLException.java:353)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1777)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257)
at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:479)
at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:367)
at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271)
at org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:93)
at org.hibernate.jdbc.AbstractBatcher.getResultSet(AbstractBatcher.java:186)
at org.hibernate.loader.Loader.getResultSet(Loader.java:1787)
at org.hibernate.loader.Loader.scroll(Loader.java:2286)
at org.hibernate.loader.criteria.CriteriaLoader.scroll(CriteriaLoader.java:89)
at org.hibernate.impl.SessionImpl.scroll(SessionImpl.java:1533)
at org.hibernate.impl.CriteriaImpl.scroll(CriteriaImpl.java:297)
at org.hibernate.impl.CriteriaImpl.scroll(CriteriaImpl.java:291)
at com.hi.openname.dao.impl.HibernateKnowledgeItemDao.loadAllCommitted(HibernateKnowledgeItemDao.java:104)
at com.hi.openname.dao.impl.HibernateKnowledgeItemDao.loadAllCommitted(HibernateKnowledgeItemDao.java:1)
Regards
Ankit
В списке pgsql-jdbc по дате отправления: