Re: Variable length varlena headers redux

Heikki Linnakangas wrote:
> Gregory Stark wrote:
> > "Tom Lane" <tgl@sss.pgh.pa.us> writes:
> >> For example it'd be easy to implement the previously-discussed design
> >> involving storing uncompressed length words in network byte order:
> >> SET_VARLENA_LEN does htonl() and VARSIZE does ntohl() and nothing else in
> >> the per-datatype functions needs to change. Another idea that we were
> >> kicking around is to make an explicit distinction between little-endian and
> >> big-endian hardware: on big-endian hardware, store the two TOAST flag bits
> >> in the MSBs as now, but on little-endian, store them in the LSBs, shifting
> >> the length value up two bits. This would probably be marginally faster than
> >> htonl/ntohl depending on hardware and compiler intelligence, but either way
> >> you get to guarantee that the flag bits are in the physically first byte,
> >> which is the critical thing needed to be able to tell the difference between
> >> compressed and uncompressed length values.
> > 
> > Actually I think neither htonl nor bitshifting the entire 4-byte word is going
> > to really work here. Both will require 4-byte alignment. Instead I think we
> > have to access the length byte by byte as a (char*) and do arithmetic. Since
> > it's the pointer being passed to VARSIZE that isn't too hard, but it might
> > perform poorly.
> 
> We would still require all datums with a 4-byte header to be 4-byte 
> aligned, right? When reading, you would first check if it's a compressed 
> or uncompressed header. If compressed, read the 1 byte header, if 
> uncompressed, read the 4-byte header and do htonl or bitshifting. No 
> need to do htonl or bitshifting on unaligned datums.

I am not sure how to handle the alignment issue.  If we require 1-byte
headers to be 4-byte aligned, we lose a lot of the benefits of the
1-byte header.

--  Bruce Momjian  <bruce@momjian.us>          http://momjian.us EnterpriseDB
http://www.enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +