Re: Pluggable toaster

Hi hackers!

Now cfbot is happy, but there were warnings due to recent changes in PointerGetDatum function, so here's corrected patchset.

Patchset consists of:

v20-0001-toaster-interface.patch - Pluggable TOAST API interface along with reference TOAST mechanics - new API is introduced but

reference TOAST is still unchanged;

v20-0002-toaster-default.patch - Default TOAST re-implemented using Toaster API - reference TOAST is re-implemented via new API;
v20-0003-toaster-docs.patch - Pluggable TOAST API documentation package

Actual GitHub branch resides at

https://github.com/postgrespro/postgres/tree/toasterapi_clean

On Tue, Oct 4, 2022 at 1:02 AM Nikita Malakhov <hukutoc@gmail.com> wrote:

Hi hackers!

Cfbot failed in meson build with previous patchsets, so I've rebased them onto the latest master and added necessary meson build info.

Patchset consists of:

v19-0001-toaster-interface.patch - Pluggable TOAST API interface along with reference TOAST mechanics - new API is introduced but

reference TOAST is still unchanged;

v19-0002-toaster-default.patch - Default TOAST re-implemented using Toaster API - reference TOAST is re-implemented via new API;
v19-0003-toaster-docs.patch - Pluggable TOAST API documentation package

Actual GitHub branch resides at

https://github.com/postgrespro/postgres/tree/toasterapi_clean

On Tue, Sep 27, 2022 at 12:26 AM Nikita Malakhov <hukutoc@gmail.com> wrote:

Hi,

Meson build for the patchset failed, meson build files attached and README/Doc package

reworked with more detailed explanation of virtual function table along with other corrections.

On Sun, Sep 25, 2022 at 1:41 AM Nikita Malakhov <hukutoc@gmail.com> wrote:

Hi hackers!

Last patchset has an invalid patch file - v16-0003-toaster-docs.patch. Here's corrected patchset,

sorry for the noise.

On Sat, Sep 24, 2022 at 3:50 PM Nikita Malakhov <hukutoc@gmail.com> wrote:

Hi hackers!

Cfbot is still not happy with the patchset, so I'm attaching a rebased one, rebased onto the current

master (from today). The third patch contains documentation package, and the second one contains large 

README.toastapi file providing additional in-depth docs for developers.

Comments would be greatly appreciated.

Again, after checking patch sources I have a strong opinion that it needs some refactoring - 

move all files related to TOAST implementation into new folder /backend/access/toast where

Generic (default) Toaster resides.

Patchset consists of:

v16-0001-toaster-interface.patch - Pluggable TOAST API interface along with reference TOAST mechanics;

v16-0002-toaster-default.patch - Default TOAST re-implemented using Toaster API;
v16-0003-toaster-docs.patch - Pluggable TOAST API documentation package

Actual GitHub branch resides at

https://github.com/postgrespro/postgres/tree/toasterapi_clean

On Fri, Sep 23, 2022 at 10:54 PM Nikita Malakhov <hukutoc@gmail.com> wrote:

Hi hackers!

Cfbot is not happy with previous patchset, so I'm attaching new one, rebased onto current master

(15b4). Also providing patch with documentation package, and the second one contains large 

README.toastapi file providing additional in-depth docs for developers.

Comments would be greatly appreciated.

Also, after checking patch sources I have a strong opinion that it needs some refactoring - 

move all files related to TOAST implementation into new folder /backend/access/toast where

Generic (default) Toaster resides.

Patchset consists of:

v15-0001-toaster-interface.patch - Pluggable TOAST API interface along with reference TOAST mechanics;

v15-0002-toaster-default.patch - Default TOAST re-implemented using Toaster API;
v15-0003-toaster-docs.patch - Pluggable TOAST API documentation package

On Tue, Sep 13, 2022 at 7:50 PM Jacob Champion <jchampion@timescale.com> wrote:

On Mon, Sep 12, 2022 at 11:45 PM Nikita Malakhov <hukutoc@gmail.com> wrote:
> It would be more clear for complex data types like JSONB, where developers could
> need some additional functionality to work with internal representation of data type,
> and its full potential is revealed in our JSONB toaster extension. The JSONB toaster
> is still in development but we plan to make it available soon.

Okay. It'll be good to have that, because as it is now it's hard to
see the whole picture.

> On installing dummy_toaster contrib: I've just checked it by making a patch from commit
> and applying onto my clone of master and 2 patches provided in previous email without
> any errors and sll checks passed - applying with git am, configure with debug, cassert,
> depend and enable-tap-tests flags and run checks.
> Please advice what would cause such a behavior?

I don't think the default pg_upgrade tests will upgrade contrib
objects (there are instructions in src/bin/pg_upgrade/TESTING that
cover manual dumps, if you prefer that method). My manual steps were
roughly

    =# CREATE EXTENSION dummy_toaster;
    =# CREATE TABLE test (t TEXT
            STORAGE external
            TOASTER dummy_toaster_handler);
    =# \q
    $ initdb -D newdb
    $ pg_ctl -D olddb stop
    $ pg_upgrade -b <install path>/bin -B <install path>/bin -d
./olddb -D ./newdb

(where <install path>/bin is on the PATH, so we're using the right binaries).

Thanks,
--Jacob

--

Regards,

Nikita Malakhov

Postgres Professional 

https://postgrespro.ru/

--

Regards,

Nikita Malakhov

Postgres Professional 

https://postgrespro.ru/

--

Regards,

Nikita Malakhov

Postgres Professional 

https://postgrespro.ru/

--

Regards,

Nikita Malakhov

Postgres Professional 

https://postgrespro.ru/

--

Regards,

Nikita Malakhov

Postgres Professional 

https://postgrespro.ru/

--

Regards,

Nikita Malakhov

Postgres Professional 

https://postgrespro.ru/