How to "unique-ify" HUGE table?
| От | Kynn Jones |
|---|---|
| Тема | How to "unique-ify" HUGE table? |
| Дата | |
| Msg-id | c2350ba40812230925jb50fed7h3dc58cc311888c7f@mail.gmail.com обсуждение исходный текст |
| Ответы |
Re: How to "unique-ify" HUGE table?
Re: How to "unique-ify" HUGE table? Re: How to "unique-ify" HUGE table? |
| Список | pgsql-performance |
Hi everyone!
I have a very large 2-column table (about 500M records) from which I want to remove duplicate records.
I have tried many approaches, but they all take forever.
The table's definition consists of two short TEXT columns. It is a temporary table generated from a query:
CREATE TEMP TABLE huge_table AS SELECT x, y FROM ... ;
Initially I tried
CREATE TEMP TABLE huge_table AS SELECT DISTINCT x, y FROM ... ;
but after waiting for nearly an hour I aborted the query, and repeated it after getting rid of the DISTINCT clause.
Everything takes forever with this monster! It's uncanny. Even printing it out to a file takes forever, let alone creating an index for it.
Any words of wisdom on how to speed this up would be appreciated.
TIA!
Kynn
В списке pgsql-performance по дате отправления: