Re: Re: Backup and Recovery

On Thu, Jul 05, 2001 at 02:27:01PM +0200, Zeugswetter Andreas SB wrote:
> 
> > Also, isn't the WAL format rather bulky to archive hours and hours of?
> 
> If it were actually too bulky, then it needs to be made less so, since
> that directly affects overall performance :-) 

ISTM that WAL record size trades off against lots of things, including 
(at least) complexity of recovery code, complexity of WAL generation 
code, usefulness in fixing corrupt table images, and processing time
it would take to produce smaller log entries.  

Complexity is always expensive, and CPU time spent "pre-sync" is a lot
more expensive than time spent in background.  That is, time spent
generating the raw log entries affects latency and peak capacity, 
where time in background mainly affects average system load.

For a WAL, the balance seems to be far to the side of simple-and-bulky.
For other uses, the balance is sure to be different.

> > > I would expect high-level transaction redo records to be much more
> > > compact; mixed into the WAL, such records shouldn't make the WAL
> > > grow much faster.
> 
> All redo records have to be at the tuple level, so what higher-level
> are you talking about ? (statement level redo records would not be
> able to reproduce the same resulting table data (keyword: transaction
> isolation level)) 

Statement-level redo records would be nice, but as you note they are 
rarely practical if done by the database.

Redo records that contain that contain whole blocks may be much bulkier
than records of whole tuples.  Redo records of whole tuples may be much 
bulkier than those that just identify changed fields.

Bulky logs mean more-frequent snapshot backups, and bulky log formats 
are less suitable for network transmission, and therefore less useful 
for replication.  Smaller redo records take more processing to generate, 
but that processing can be done off-line, and the result saves other 
costs.

Nathan Myers
ncm@zembu.com