Re: On columnar storage

On 6/14/15 10:22 AM, Alvaro Herrera wrote:
>> >To me, it feels like there are two different features here that would
>> >be better separated.  First, there's the idea of having a table that
>> >gets auto-joined to other tables whenever you access it, so that the
>> >user sees one really wide table but really the data is segregated by
>> >column groups under the hood.  That's a neat idea.
> Thanks.  (It also seems pretty tricky to implement.)

I look at it as a form of vertical partitioning; the big difference 
being whether you normalize the columns out or not (or to use data 
warehouse parlance, slow vs fast changing dimensions).

Perhaps it would be useful to vet this out as a userspace extension 
first since that would presumably be much easier. I believe we actually 
have all the backend infrastructure that would be needed for this now 
that views are smart enough to exclude tables that aren't referenced at 
all. I suspect that even a 'dumb userspace' approach would still expose 
a lot of the planner problems we'll run into (join explosion and 
filtering through the join come to mind).

Related to idea of an 'auto join', I do wish we had the ability to 
access columns in a referenced FK table from a referring key; something 
like SELECT customer_id.first_name FROM invoice (which would be 
translated to SELECT first_name FROM invoice JOIN customer USING( 
customer_id )).
-- 
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Data in Trouble? Get it in Treble! http://BlueTreble.com