Re: Enabling Checksums

On 3/8/13 3:38 AM, Heikki Linnakangas wrote:
> See https://www.kernel.org/doc/Documentation/block/data-integrity.txt

That includes an interesting comment that's along the lines of the MySQL 
checksum tests already mentioned:

"The 16-bit CRC checksum mandated by both the SCSI and SATA specs
is somewhat heavy to compute in software.  Benchmarks found that
calculating this checksum had a significant impact on system
performance for a number of workloads.  Some controllers allow a
lighter-weight checksum to be used when interfacing with the operating
system.  Emulex, for instance, supports the TCP/IP checksum instead."

The TCP/IP checksum spec is at https://tools.ietf.org/html/rfc793 ; its 
error detection limitations are described at 
http://www.noahdavids.org/self_published/CRC_and_checksum.html ; and a 
good article about optimizing its code is at 
http://www.locklessinc.com/articles/tcp_checksum/  I'll take a longer 
look at whether it's an improvement on the Fletcher-16 used by the 
current patch.  All of these 16 bit checksums are so much better than 
nothing.  I don't think some shift toward prioritizing computation speed 
over detection rate is a problem.  In the long run really sensitive 32 
bit checksums will become more practical.

As Heikki pointed out, the direction this whole area seems to be going 
is that one day you might get checksums all the way from application to 
hardware.  That's another possible future where having some field tested 
checksum feature in the database will be valuable.

-- 
Greg Smith   2ndQuadrant US    greg@2ndQuadrant.com   Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com