Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

Поиск
Список
Период
Сортировка
От Nikita Malakhov
Тема Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)
Дата
Msg-id CAN-LCVPyWKAgPMPkfkoe6_H3uG1ue1a6YbK0Zsn_UEL_CM-3QA@mail.gmail.com
обсуждение исходный текст
Ответ на Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)  (Damir Belyalov <dam.bel07@gmail.com>)
Список pgsql-hackers
Hi Damir!

Your work looks like a very promising feature for production systems,
where data often needs to be loaded from external sources.

I've looked over the discussion and want to make a proposal -
when we load a bunch of records in database it does not make sense
to output errors to command output, and does not make sense to limit
error output to any number at all, because if we decided to load data
anyway - we would want to have a list (a file) with all records that were
discarded because of errors, with related error information, to, say,
deal with errors and process these records later. It looks like a reasonable
addition to your patch.

As a command output some limited number of error messages has much
less meaning than overall stats - records processed, records loaded,
records discarded, total number of errors.

For example you can look the Oracle SQL Loader feature, I hope this could
give some ideas for further improvements.

On Wed, Nov 2, 2022 at 11:46 AM Damir Belyalov <dam.bel07@gmail.com> wrote:
Updated the patch:
- Optimized and simplified logic of IGNORE_ERRORS
- Changed variable names to more understandable ones
- Added an analogue of MAX_BUFFERED_BYTES for safe buffer


Regards,
Damir Belyalov
Postgres Professional


--
Regards,
Nikita Malakhov
Postgres Professional 

В списке pgsql-hackers по дате отправления:

Предыдущее
От: Melih Mutlu
Дата:
Сообщение: Re: wake up logical workers after ALTER SUBSCRIPTION
Следующее
От: vignesh C
Дата:
Сообщение: Re: Support logical replication of DDLs