Re: Fastest method to insert data.

On Sat, 20 Apr 2002, Dennis R. Gesker wrote:

> In the sequences of actually transferring the data I encountered some
> memory problems when using the .addBatch() methods. I probably should
> have expected this since the tables I'm seeking to draw data from are
> kind of large. I was thinking that using the .addBatch() would be a good
> approach since I could treat the whole batch as a transaction helping to
> ensure that I pulled everything I intended.

You'll need to issue executeBatch() every once in a while.  My current
approach is to executeBatch() for every thousand rows.  I've tried many
things, this is the one that proved to be the fastest.

But I think that this is still much slower than it needs to be.  COPY
TABLE is still much faster.

> Now, if there were classes that could GULP whole tables from a database
> (PG., MS or otherwise) at one shot and recreate these tables in PG. this
> would be great!
>
> Sam: Is the direction in which you were thinking?

Yes.  The Sybase SQL server has a complete separate set of APIs that are
designed to quickly upload a bunch of data to a table.  Sybase's bulk-copy
API is very similar to addBatch(), executeBatch() except that there's no
SQL involved.  You just specify the table, then start feeding it rows.