albertwang schrieb am 23.06.2017 um 02:08:
> I met the same problem. Our company uses PostgreSQL as the OLTP system. I am
> working on an ETL project which reads data from a PostgreSQL db, transforms
> the data and then load the data into another PostgreSQL db. The ETL tool
> uses JDBC.
>
> My problem is that the ETL won't start to transform the data before the
> whole data result set has been loaded into memory.
>
> I would like to do the data transforming and loading while extracting data.
>
> Do you know if there is any progress reagrding the streaming result set? As
> I know, MySQL does support it.
Not sure what you mean with "streaming resultset", but if you don't want the driver to buffer the complete result, then
thereis a solution to it:
retrieving based on a cursor.
See the documentation for details:
https://jdbc.postgresql.org/documentation/head/query.html#query-with-cursor
Thomas