Обсуждение: Large files in main/base

Поиск
Список
Период
Сортировка

Large files in main/base

От
"Henry, Frank"
Дата:
Hello everyone,

We had a problem with one of our servers and had noticed that the
postgres/8.3/main folder had become quite large (>650mb).
PostgreSQL confirmed this via a query but when I queried the size of the
tables I was barely reaching 3mb.

After looking around, I found one file with about 580 mb. From the
contents I have the feeling it is some kind of changelog or similar.

My Questions:
What is it?
How can I reduce it?
How can I prevent it from reaching that size again?

Thanks

Frank


DISCLAIMER:
Unless indicated otherwise, the information contained in this message is privileged and confidential, and is intended
onlyfor the use of the addressee(s) named above and others who have been specifically authorized to receive it. If you
arenot the intended recipient, you are hereby notified that any dissemination, distribution or copying of this message
and/orattachments is strictly prohibited. The company accepts no liability for any damage caused by any virus
transmittedby this email. Furthermore, the company does not warrant a proper and complete transmission of this
information,nor does it accept liability for any delays. If you have received this message in error, please contact the
senderand delete the message. Thank you. 

Re: Large files in main/base

От
Scott Marlowe
Дата:
On Fri, Jul 9, 2010 at 7:31 AM, Henry, Frank <frank.henry@barco.com> wrote:
> Hello everyone,
>
> We had a problem with one of our servers and had noticed that the
> postgres/8.3/main folder had become quite large (>650mb).
> PostgreSQL confirmed this via a query but when I queried the size of the
> tables I was barely reaching 3mb.
>
> After looking around, I found one file with about 580 mb. From the
> contents I have the feeling it is some kind of changelog or similar.

And it is?  So close to telling us.

>
> My Questions:
> What is it?

Why don't you tell us?

Re: Large files in main/base

От
Tom Lane
Дата:
Scott Marlowe <scott.marlowe@gmail.com> writes:
> On Fri, Jul 9, 2010 at 7:31 AM, Henry, Frank <frank.henry@barco.com> wrote:
>> What is it?

> Why don't you tell us?

The exact name of the file might be useful information, too.

            regards, tom lane

Re: Large files in main/base

От
"Joshua D. Drake"
Дата:
On Fri, 2010-07-09 at 15:31 +0200, Henry, Frank wrote:
> Hello everyone,
>
> We had a problem with one of our servers and had noticed that the
> postgres/8.3/main folder had become quite large (>650mb).
> PostgreSQL confirmed this via a query but when I queried the size of the
> tables I was barely reaching 3mb.
>
> After looking around, I found one file with about 580 mb. From the
> contents I have the feeling it is some kind of changelog or similar.

Name of file?
Version of PostgreSQL?

JD




Re: Large files in main/base

От
Scott Marlowe
Дата:
On Fri, Jul 9, 2010 at 7:31 AM, Henry, Frank <frank.henry@barco.com> wrote:
> Hello everyone,
>
> We had a problem with one of our servers and had noticed that the
> postgres/8.3/main folder had become quite large (>650mb).
> PostgreSQL confirmed this via a query but when I queried the size of the
> tables I was barely reaching 3mb.
>
> After looking around, I found one file with about 580 mb. From the
> contents I have the feeling it is some kind of changelog or similar.

pg_clog?  If that's growing you've got a possible problem with a long
running query.

Re: Large files in main/base

От
"Neuber, Dirk"
Дата:

>> What is it?

> Why don't you tell us?

> The exact name of the file might be useful information, too.

It is : postgres/8.3/main/base/16385/2613

Thanks.
Ciao Dirk.

DISCLAIMER:
Unless indicated otherwise, the information contained in this message is privileged and confidential, and is intended only for the use of the addressee(s) named above and others who have been specifically authorized to receive it. If you are not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this message and/or attachments is strictly prohibited. The company accepts no liability for any damage caused by any virus transmitted by this email. Furthermore, the company does not warrant a proper and complete transmission of this information, nor does it accept liability for any delays. If you have received this message in error, please contact the sender and delete the message. Thank you.

Re: Large files in main/base

От
Guillaume Lelarge
Дата:
Le 10/07/2010 12:43, Neuber, Dirk a écrit :
>>> What is it?
>
>> Why don't you tell us?
>
>> The exact name of the file might be useful information, too.
>
> It is : postgres/8.3/main/base/16385/2613
>

This is a system catalog, pg_largeobject, which holds binary objects. If
you use Large Objects, no wonder this table could be really big.


--
Guillaume
 http://www.postgresql.fr
 http://dalibo.com

Re: Large files in main/base

От
Tom Lane
Дата:
Guillaume Lelarge <guillaume@lelarge.info> writes:
> Le 10/07/2010 12:43, Neuber, Dirk a �crit :
>>> The exact name of the file might be useful information, too.
>>
>> It is : postgres/8.3/main/base/16385/2613

> This is a system catalog, pg_largeobject, which holds binary objects. If
> you use Large Objects, no wonder this table could be really big.

Also worth noting is that there's no automatic deletion mechanism for
large objects.  It could be that the space is being eaten by LOs that
aren't referenced anymore.  If so, you should think about applying
contrib/lo and/or contrib/vacuumlo to manage your LOs.

            regards, tom lane

Re: Large files in main/base

От
"Neuber, Dirk"
Дата:
> >> It is : postgres/8.3/main/base/16385/2613
>
> > This is a system catalog, pg_largeobject, which holds
> binary objects. If
> > you use Large Objects, no wonder this table could be really big.
>
> Also worth noting is that there's no automatic deletion mechanism for
> large objects.  It could be that the space is being eaten by LOs that
> aren't referenced anymore.  If so, you should think about applying
> contrib/lo and/or contrib/vacuumlo to manage your LOs.
>

Thanks a lot.
vacuumlo did the job. Indeed there were more the 100.000 orphaned LOs in
pg_largeobject.
We have to check how we can avoid this by using contrib/lo or in general
without using LOs.
One last question. After using vacuumlo the file size of 16385/2613 is
still the same as before.
It seems the content has been removed but the file itself hasn't been
compacted.
Is there an option or tool which can do this as well ?

Ciao Dirk.


DISCLAIMER:
Unless indicated otherwise, the information contained in this message is privileged and confidential, and is intended
onlyfor the use of the addressee(s) named above and others who have been specifically authorized to receive it. If you
arenot the intended recipient, you are hereby notified that any dissemination, distribution or copying of this message
and/orattachments is strictly prohibited. The company accepts no liability for any damage caused by any virus
transmittedby this email. Furthermore, the company does not warrant a proper and complete transmission of this
information,nor does it accept liability for any delays. If you have received this message in error, please contact the
senderand delete the message. Thank you. 

Re: Large files in main/base

От
Tom Lane
Дата:
"Neuber, Dirk" <dirk.neuber@barco.com> writes:
> One last question. After using vacuumlo the file size of 16385/2613 is
> still the same as before.
> It seems the content has been removed but the file itself hasn't been
> compacted.
> Is there an option or tool which can do this as well ?

Yeah, you still need to apply VACUUM FULL to bring the physical file
size down.  (Ordinary VACUUM mostly just recycles space within the
file --- it can truncate the file if the end pages are empty, but
it won't move data around to make that possible.)

            regards, tom lane