Re: Proposal: Incremental Backup

On Wed, Aug  6, 2014 at 01:15:32PM -0300, Claudio Freire wrote:
> On Wed, Aug 6, 2014 at 12:20 PM, Bruce Momjian <bruce@momjian.us> wrote:
> >
> > Well, for file-level backups we have:
> >
> >         1) use file modtime (possibly inaccurate)
> >         2) use file modtime and checksums (heavy read load)
> >
> > For block-level backups we have:
> >
> >         3) accumulate block numbers as WAL is written
> >         4) read previous WAL at incremental backup time
> >         5) read data page LSNs (high read load)
> >
> > The question is which of these do we want to implement?  #1 is very easy
> > to implement, but incremental _file_ backups are larger than block-level
> > backups.  If we have #5, would we ever want #2?  If we have #3, would we
> > ever want #4 or #5?
> 
> You may want to implement both #3 and #2. #3 would need a config
> switch to enable updating the bitmap. That would make it optional to
> incur the I/O cost of updating the bitmap. When the bitmap isn't
> there, the backup would use #2. Slow, but effective. If slowness is a
> problem for you, you enable the bitmap and do #3.
> 
> Sounds reasonable IMO, and it means you can start by implementing #2.

Well, Robert Haas had the idea of a separate process that accumulates
the changed WAL block numbers, making it low overhead.  I question
whether we need #2 just to handle cases where they didn't enable #3
accounting earlier.  If that is the case, just do a full backup and
enable #3.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +