Re: Extensible storage manager API - smgr hooks

Anastasia Lubennikova писал 2021-06-30 00:49:
> Hi, hackers!
> 
> Many recently discussed features can make use of an extensible storage
> manager API. Namely, storage level compression and encryption [1],
> [2], [3], disk quota feature [4], SLRU storage changes [5], and any
> other features that may want to substitute PostgreSQL storage layer
> with their implementation (i.e. lazy_restore [6]).
> 
> Attached is a proposal to change smgr API to make it extensible.  The
> idea is to add a hook for plugins to get control in smgr and define
> custom storage managers. The patch replaces smgrsw[] array and smgr_sw
> selector with smgr() function that loads f_smgr implementation.
> 
> As before it has only one implementation - smgr_md, which is wrapped
> into smgr_standard().
> 
> To create custom implementation, a developer needs to implement smgr
> API functions
>     static const struct f_smgr smgr_custom =
>     {
>         .smgr_init = custominit,
>         ...
>     }
> 
> create a hook function
> 
>    const f_smgr * smgr_custom(BackendId backend, RelFileNode rnode)
>   {
>       //Here we can also add some logic and chose which smgr to use
> based on rnode and backend
>       return &smgr_custom;
>   }
> 
> and finally set the hook:
>     smgr_hook = smgr_custom;
> 
> [1]
> https://www.postgresql.org/message-id/flat/11996861554042351@iva4-dd95b404a60b.qloud-c.yandex.net
> [2]
> https://www.postgresql.org/message-id/flat/272dd2d9.e52a.17235f2c050.Coremail.chjischj%40163.com
> [3] https://postgrespro.com/docs/enterprise/9.6/cfs
> [4]
> https://www.postgresql.org/message-id/flat/CAB0yre%3DRP_ho6Bq4cV23ELKxRcfhV2Yqrb1zHp0RfUPEWCnBRw%40mail.gmail.com
> [5]
> https://www.postgresql.org/message-id/flat/20180814213500.GA74618%4060f81dc409fc.ant.amazon.com
> [6]
> https://wiki.postgresql.org/wiki/PGCon_2021_Fun_With_WAL#Lazy_Restore
> 
> --
> 
> Best regards,
> Lubennikova Anastasia

Good day, Anastasia.

I also think smgr should be extended with different implementations 
aside of md.
But which way concrete implementation will be chosen for particular 
relation?
I believe it should be (immutable!) property of tablespace, and should 
be passed
to smgropen. Patch in current state doesn't show clear way to distinct 
different
implementations per relation.

I don't think patch should be that invasive. smgrsw could pointer to
array instead of static array as it is of now, and then reln->smgr_which
will remain with same meaning. Yep it then will need a way to select 
specific
implementation, but something like `char smgr_name[NAMEDATALEN]` field 
with
linear search in (i believe) small smgrsw array should be enough.

Maybe I'm missing something?

regards,
Sokolov Yura.