Re: Add new COPY option REJECT_LIMIT

On 2024/07/19 22:03, Fujii Masao wrote:
> 
> 
> On 2024/07/17 22:21, torikoshia wrote:
>> On 2024-07-03 02:07, Fujii Masao wrote:
>>> However, if we support REJECT_LIMIT, I'm not sure if the ON_ERROR option is still necessary.
>>
>> I remembered another reason for the necessity of ON_ERROR.
>>
>> ON_ERROR defines how to behave when encountering an error and it just accepts 'ignore' and 'stop' currently, but is
expectedto support other options such as saving details of errors to a table[1].
 
> 
> Wouldn't it be better to separate the option specifying where
> error details are output from the ON_ERROR option
> (which determines behavior when encountering errors)?
> "table" seems valid for both ON_ERROR=ignore and ON_ERROR=stop.

I still find it odd to accept "table" as a value for ON_ERROR. However,
"set_to_null" or "replace-column" proposed in [1] seem valid for
ON_ERROR. So, I'm okay with keeping the ON_ERROR option.


> On my second thought, whatever value ON_ERROR is specified(e.g. ignore, stop, table), it seems fine to use
REJECT_LIMIT.
> I feel REJECT_LIMIT has both "ignore" and "stop" characteristics, meaning it ignores errors until it reaches
REJECT_LIMITand stops when it exceeds the REJECT_LIMIT.
 

ON_ERROR specifies how to handle errors, and "stop" means to fail
the command. So, if ON_ERROR=stop, REJECT_LIMIT should have no effect,
and the command should fail immediately upon encountering an error.

As in your original proposal, I now think REJECT_LIMIT should only
apply when ON_ERROR=ignore. The command would ignore errors and
continue processing, but if the number of errors exceeds REJECT_LIMIT,
the command should fail. Thought?

BTW if "set_to_null" is supported someday, REJECT_LIMIT can also
apply. The command would cinsert NULL into the target table upon
encountering errors and continue, but fail if the number of errors
exceed REJECT_LIMIT.

Regards,

[1] https://www.postgresql.org/message-id/flat/CAKFQuwawy1e6YR4S=j+y7pXqg_Dw1WBVrgvf=BP3d1_aSfe_+Q@mail.gmail.com

-- 
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION