Re: Zedstore - compressed in-core columnar storage

We've continued hacking on Zedstore, here's a new patch version against 
current PostgreSQL master (commit f1bf619acdf). If you want to follow 
the development in real-time, we're working on this branch: 
https://github.com/greenplum-db/postgres/tree/zedstore

If you want to do performance testing with this, make sure you configure 
with the --with-lz4 option. Otherwise, you'll get pglz compression, 
which is *much* slower.


Major TODOs:

* Make it crash-safe, by WAL-logging.

* Teach the planner and executor to pass down the list of columns 
needed. Currently, many plans will unnecessarily fetch columns that are 
not needed.

* Make visibility checks against the TID tree in index-only scans.

* zedstore-toast pages are currently leaked, so you'll get a lot of 
bloat if you delete/update rows with large datums

* Use the UNDO framework that's been discussed on another thread. 
There's UNDO-logging built into zedstore at the moment, but it's not 
very optimized.

* Improve free space management. Pages that become empty are currently 
recycled, but space on pages that are not completely empty is not not 
reused, and half-empty pages are not merged.

* Implement TID recycling. Currently, TIDs are allocated in increasing 
order, and after all 2^48 TIDs have been used, even if the rows have 
been deleted since, no more ruples can be inserted.

- Heikki