Re: Columnar format export in Postgres

Hi,

In <CAH5mb9_YW76_0xBU2T4C7HF33b+b2w3QBtV50_ZZMy8SA8ChjA@mail.gmail.com>
  "Columnar format export in Postgres" on Wed, 12 Jun 2024 22:26:30 +0530,
  Sushrut Shivaswamy <sushrut.shivaswamy@gmail.com> wrote:

> I have been working on adding support for columnar format export to
> Postgres to speed up analytics queries.

FYI: I'm proposing making COPY format extendable:

* https://www.postgresql.org/message-id/flat/20231204.153548.2126325458835528809.kou@clear-code.com
* https://commitfest.postgresql.org/48/4681/

If it's accepted, we can implement extensions for COPY
FORMAT arrow and COPY FORMAT parquet. With these extensions,
we can use file_fdw to read Apache Arrow and Apache Parquet
file because file_fdw is based on COPY FROM:
https://www.postgresql.org/docs/current/file-fdw.html

If you're interested in this proposal, you can review the
latest proposed patch set to proceed this proposal.


>  - Reduce memory consumption when exporting table data to columnar format

The above COPY support will help this.


Thanks,
-- 
kou