Re: [HACKERS] Problems with >2GB tables on Linux 2.0

On Mon, 8 Feb 1999, Tom Lane wrote:

> Bruce Momjian <maillist@candle.pha.pa.us> writes:
> >> However, I'm using John's suggestion of reducing the file size a lot more,
> >> to ensure we don't hit any math errors, etc. So the max file size is about
> >> 1.6Gb.
> 
> > I can imagine people finding that strange.  It it really needed.  Is
> > there some math that could overflow with a larger value?
> 
> Well, that's the question all right --- are you sure that there's not?
> I think "max - 1 blocks" is pushing it, since code that computes
> something like "the byte offset of the block after next" would fail.
> Even if there isn't any such code today, it seems possible that there
> might be someday.
> 
> I'd be comfortable with 2 billion (2000000000) bytes as the filesize
> limit, or Andreas' proposal of 1Gb.

I'm starting to like Andreas' proposal as the new default.

> I also like the proposals to allow the filesize limit to be configured
> even lower to ease splitting huge tables across filesystems.
> 
> To make that work easily, we really should adopt a layout where the data
> files don't all go in the same directory.  Perhaps the simplest is:
> 
> * First or only segment of a table goes in top-level data directory,
>   same as now.
> 
> * First extension segment is .../data/1/tablename.1, second is
>   .../data/2/tablename.2, etc.  (Using numbers for the subdirectory
>   names prevents name conflict with ordinary tables.)

How about dropping the suffix, so you would have:
.../data/2/tablename

Doing that doesn't mean having to increase the filename buffer size, just
the format and arg order (from %s.%d to %d/%s).

I'd think we could add a test when the new segment is created for the
symlink/directory. If it doesn't exist, then create it. Otherwise a poor
unsuspecting user would have their database fall over, not realising where
the error is.

Peter

--       Peter T Mount peter@retep.org.uk     Main Homepage: http://www.retep.org.uk
PostgreSQL JDBC Faq: http://www.retep.org.uk/postgresJava PDF Generator: http://www.retep.org.uk/pdf