Re: [ERROR] Copy from CSV fails due to memory error.

On 1/19/22 14:01, Kostas Chasialis wrote:
> Hey.
> 
> I am facing an issue when I try to run the following command
> 
> COPY <table_name> FROM <file> WITH DELIMITER E',’;
> 
> This file, is rather large, it's around 178GBs.
> 
> When I try to run this COPY command I get the following error:
> 
> ERROR:  out of memory
> DETAIL:  Failed on request of size 2048 in memory context "AfterTriggerEvents".
> CONTEXT:  COPY ssbm300_lineorder, line 50796791
> 
> Clearly a memory allocation function is failing but I have no clue how to fix it.
> 
> I have tried experimenting with shared_buffers value in postgresql.conf file but after searching a bit I quickly
realizedthat I do not know what I am doing there so I left it with default value. Same with work_mem value.
 
> 
> Did you face this issue before? Can you help me resolve it?
> 

Well, it's clearly related to "after" triggers - do you have anything 
such triggers on the table? AFAIK it might be related to deferred 
constraints (like unique / foreign keys). Do you have anything like that?

If yes, I guess the only solution is to make the constraints not 
deferred or split the copy into smaller chunks.

regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company