Re: [HACKERS] Pluggable storage

Hi,

On 06/26/2017 05:18 PM, Alexander Korotkov wrote:
> Hackers,
> 
> I see that design question for PostgreSQL pluggable storages is very 
> hard.

IMHO it's mostly expected to be hard.

Firstly, PostgreSQL is a mature product with many advanced features, and 
reworking a low-level feature without breaking something on top of it is 
hard by definition.

Secondly, project policies and code quality requirements set the bar 
very high too, I think.
> BTW, I think it worth analyzing existing use-cases of pluggable
> storages.  I think that the most famous DBMS with pluggable storage API
> is MySQL. This why I decided to start with it. I've added
> MySQL/MariaDB section on wiki page.
> https://wiki.postgresql.org/wiki/Future_of_storage#MySQL.2FMariaDB
> It appears that significant part of MySQL storage engines are misuses.  
> MySQL lacks of various features like FDWs or writable views and so on.  
> This is why developers created a lot of pluggable storages for that 
> purposes.  We definitely don't want something like this in PostgreSQL 
> now.  I created special resume column where I expressed whether it
> would be nice to have something like this table engine in PostgreSQL.
> 

I don't want to discourage you, but I'm not sure how valuable this is.

I agree it's valuable to have a an over-view of use cases for pluggable 
storage, but I don't think we'll get that from looking at MySQL. As you 
noticed, most of the storage engines are misuses, so it's difficult to 
learn anything valuable from them. You can argue that using FDWs to 
implement alternative storage engines is a misuse too, but at least that 
gives us a valid use case (columnar storage implemented using FDW).

If anything, the MySQL storage engines should serve as a cautionary tale 
how not to do things - there's also a plenty of references in the MySQL 
"Restrictions and Limitations" section of the manual:
  https://downloads.mysql.com/docs/mysql-reslimits-excerpt-5.7-en.pdf

regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services