> I have a Java program that is trying to import over 8 million
> rows from a text file.
>
> When the program starts up, it runs fine, but after a while
> it begins to crawl.
>
> This is usually symptomatic of a machine that started swapping,
> but take a look at the attached output from top.
>
> I am using a LOT of memory, but the swap is still very
> low. I don't think the machine should be paging to disk.
>
> The CPU states show that the cpus are spending most of their time
> in an idle state. The process list shows the postmaster and java
> processes are less than 10 percent the CPU cycles.
>
> Having said that, the load average is still about 2.
>
> Any ideas what could be going on?
Your question is a good one for the performance list. From what I've read there, there are a
couple of strategies that you can use to improve performance.
1) if you can roll large numbers of insert statements (say 1K to 10K ins/trans) in to single
transactions, performance should be helped.
2) If this is a one time insert, you may consider TEMPORARILY turning off fsync and greatly
increase your check point segment size for the import.
Regards,
Richard Broersma Jr.