Re: [PATCHES] Fix for large file support (nonsegment mode support)

Peter Eisentraut napsal(a):
> Zdenek Kotala wrote:
>> But how it was mentioned in this thread maybe
>> somethink like this "CREATE TABLESPACE name LOCATION '/my/location'
>> SEGMENTS 10GB" should good solution. If segments is not mentioned then
>> default value is used.
>
> I think you would need a tool to resegmentize a table or tablespace offline,
> usable for example when recovering a backup.

Do you mean something like strip(1) command? I don't see any usecase for
terrabytes data. You usually have a problem to find place where you can backup.

> Also, tablespace configuration information is of course also stored in a
> table.  pg_tablespace probably won't become large, but it would probably
> still need to be special-cased, along with other system catalogs perhaps.

It is true and unfortunately singularity. Same as database list which is in a
table as well, but it is stored also as a text file for startup purpose. I more
incline to use non table configuration file for tablespaces, because I don't see
any advantage to have it under MVCC control and it allow also to define storage
for pg_global and pg_default.

> An then, how to coordindate offline resegmenting and online tablespace
> operations in a crash-safe way?
>
> Another factor I just thought of is that tar, commonly used as part of a
> backup procedure, can on some systems only handle files up to 8 GB in size.
> There are supposed to be newer formats that can avoid that restriction, but
> it's not clear how widely available these are and what the incantation is to
> get at them.  Of course we don't use tar directly, but if we ever make large
> segments the default, we ought to provide some clear advice for the user on
> how to make their backups.

I think tar is OK - minimal on Solaris. See man largefile.

Default segment size still should be 1GB. If DBA makes a decision to increase
this to higher value, then it is his responsibility to find way how to process
this big files.

        Zdenek