Re: OutOfMemory when inserting stream of unknown length

On Thu, 19 Aug 2004, Mikko T. wrote:

>
> When writing a java stream of unknown length to a bytea column with the
> prepared statement setBinaryStream the jdbc drivers throws always an
> OutOfMemoryException.
>
> I'm calling the function with following parameters as I do not know the real
> length of the stream:
>
> stm.setBinaryStream(1, stream, Interger.MAX_VALUE);
>
> and it seems that somewhere along the way the jdbc driver wrongly
> assumes that the stream contains gigabytes of data and a memory
> allocation fails.

This is true of the stable driver branch, it will try to allocate a huge
amount of memory.  The development driver has the ability to stream data
to the backend, but unfortunately for you requires an exact length
argument.  The problem is that the frontend/backend protocol messages are
preceded with a length parameter.  So the options are to read the whole
stream to determine the length (which requires huge amounts of memory to
store the whole stream), or take the provided length argument at face
value and stream data to the server hoping the user was right.  In the
streaming case it's ok if the length argument is less than the length of
the stream because then we can just stop reading, but not if the stream
itself is short.

Kris Jurka