Have you tried moving the input file to the same disk as the server, to factor out the controller?
I labour under the delusion that it is through the controller one reads and writes and that there might be some slight advantage to not doing both against one drive if avoidable. Wrong again?
Well there is one way to find out:)
Might try with something less then the whole file to get come up an approximate row/sec rate.
Well things slow down over time, and lots of "too frequent"s:
Have done 500 batches in 24219 ms Have done 1000 batches in 52362 ms Have done 1500 batches in 82256 ms Have done 2000 batches in 113754 ms Have done 2500 batches in 149637 ms Have done 3000 batches in 211314 ms Have done 3500 batches in 301989 ms Have done 4000 batches in 430817 ms Have done 4500 batches in 596043 ms Have done 5000 batches in 804250 ms
where a batch is 500,000 lines. This on the java side of course.