ps.setCharacterStream() and memory usage
От | Sebastiaan van Erk |
---|---|
Тема | ps.setCharacterStream() and memory usage |
Дата | |
Msg-id | 20041029115651.GE4199@sebster.com обсуждение исходный текст |
Ответы |
Re: ps.setCharacterStream() and memory usage
Re: ps.setCharacterStream() and memory usage Re: ps.setCharacterStream() and memory usage |
Список | pgsql-jdbc |
Hi, Coming back to another problem I have with the following insert statement: (Postgres 7.4, driver build 215, jdbc 3) ps.setCharacterStream(1, reader, (int) messageFile.length()); ps.executeUpdate(); The reason I do this (using a character stream to load a text file to a TEXT field) is that I wish to avoid loading the file into memory completely. I already noticed that the postgres driver does not stream the data to the backend (I don't know if postgres actually supports this). But what is worse, is that the file actually gets copied THREE times into memory, causing my Java file to already get a java.lang.OutOfMemory error with an insert of a 10M text file (Java allocates 20M for this, since it has 2 byte chars, and the driver makes (at least) 3 copies (referenced at the same time, not allowing one to be GC'ed), making 60M). First of all, in setCharacterStream() it loads the file into a char array. Then this is cloned into a String and passed to setString. (Here one can already cause the char[] to go out of scope, at least allowing it to be cleaned up). Secondly, setString causes another clone in escapeString(). (Ideally, one could read the stream into a StringBuffer big enough to allow it to be escaped, and thereby only load the file into memory once instead of three times). Finally, the driver (possibly) keeps large objects in memory too long: for example, in setString() right after the bind, one can already do a "sbuf.setLength(0)". This way, one does not have to wait for setString (or another setter) to be called before the sbuf variable is cleared. Greetings, Sebastiaan van Erk
В списке pgsql-jdbc по дате отправления: