Re: [HACKERS] sorting big tables :(

The Hermit Hacker wrote:

> Now, as a text file, this would amount to, what...~50MB?
40M of records to produce a 50MB text file? How would you sort such a
*compressed* file? ;-)

> So, if I were to do a 'copy out' to a text file, a Unix sort and then a
> 'copy in', I would use up *less* disk space (by several orders of
> magnitude) then doing the sort inside of PostgreSQL?

Well, I think it might be optimised slightly. Am I right that postgres
uses heap (i.e. they look like tables) files during sorting? While this
is a merge sort, those files doesn't have to be a table-like files.
Certainly, they might variable length records without pages (aren't they
used sequentially). Moreover we would consider packing tape files before
writting them down if necessary. Of course it will result in some
performance dropdown. However it's better to have less performance that
being unable to sort it at all.

Last question... What's the purpose of such a big sort? If somebody gets
40M of sorted records in a result of some query, what would he do with
it? Is he going to spent next years on reading this lecture? I mean,
isn't it worth to query the database for necessary informations only and
then sort it?

Mike

--
WWW: http://www.lodz.pdi.net/~mimo  tel: Int. Acc. Code + 48 42 148340
add: Michal Mosiewicz  *  Bugaj 66 m.54 *  95-200 Pabianice  *  POLAND