Re: Status of the table access method work

On 08/04/2019 19:38, Andres Freund wrote:
> On 2019-04-08 14:53:53 +0300, Heikki Linnakangas wrote:
>> On 05/04/2019 23:25, Andres Freund wrote:
>>> - the (optional) bitmap heap scan API - that's fairly intrinsically
>>>     block based. An AM could just internally subdivide TIDs in a different
>>>     way, but I don't think a bitmap scan like we have would e.g. make a
>>>     lot of sense for an index oriented table without any sort of stable
>>>     tid.
>>
>> If an AM doesn't implement the bitmap heap scan API, what happens? Bitmap
>> scans are disabled?
> 
> Yea, the planner doesn't consider them. It just masks the index's
> amhasgetbitmap.  Seems to be the most reasonable thing to do?

Yep.

>> Even if an AM isn't block-oriented, the bitmap heap scan API still makes
>> sense as long as there's some correlation between TIDs and physical
>> location.
> 
> Yea, it could be a non-linear mapping. But I'm honestly not sure how
> many non-block oriented AMs with such a correlation there are - I mean
> you're not going to have that in say an IOT. And it'd be trivial to just
> "fake" a block mapping for an in-memory AM.

Now that Ashwin conveniently posted the ZedStore prototype we started to 
hack on [1], I'll point to that as an example :-). It stores data in a 
B-tree (or rather, multiple B-trees) on TIDs. So there's very high 
correlation between TIDs and physical locality, but it's not block oriented.

Another example would be the "LZ4 Compressed Storage Manager" that 
Nikolai envisioned recently [2]. Before we came up with the idea of 
using b-trees in ZedStore, we were actually thinking of something very 
similar to that. Although that one perhaps still counts as 
"block-oriented" as far as the bitmap heap scan API is concerned, as it 
still deals with blocks, they're just mapped to different physical 
locations.

I'm not sure how an Index-Organized-Table would work, but I think it 
would want to just get the whole bitmap, and figure out the best order 
to fetch the rows by itself.

PS. Seems that having a table AM API has opened the floodgates for 
storage ideas. Nice!

- Heikki

[1] 
https://www.postgresql.org/message-id/CALfoeiuF-m5jg51mJUPm5GN8u396o5sA2AF5N97vTRAEDYac7w@mail.gmail.com
[2] 
https://www.postgresql.org/message-id/flat/11996861554042351%40iva4-dd95b404a60b.qloud-c.yandex.net