Re: High CPU usage

On 2022-10-20 15:59, ertan.kucukoglu@1nar.com.tr wrote:
> Hello,
>
> I am using PostgreSQL v14.5 on Linux Debian 11.5. I recently observe very
> high CPU usage on my Linux system as below
>
>      PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+
> COMMAND
> 2357756 postgres  20   0 2441032   2,3g      4 S 298,7  67,9   2114:58
> Tspjzj2Z
>
> I could not find any file named Tspjzj2Z on the file system. I could not
> find PID number using below SQL

Hi,

I'm not an expert in PostgreSQL but that looks like a rogue app, if 
you're lucky just a miner running as the prostgres user, likely the 
result of a postgres RCE exploited successfully... The more worring case 
would be a program exfiltrating and/or encrypting the database in a 
ransomware attack.

The executable has most likely been removed to hide traces, or cleaned 
up automatically from ex. /tmp, however if the process is still running 
you should be able to cat the executable, and any other open files, 
directly from /proc/<pid>/ (look for exe and fd/*).

I strongly recommend you check other postgres servers you have, make a 
copy of any process file found (for later investigation), then isolate 
or shutdown these servers and proceed with a proper investigation from a 
livecd or revovery OS.

> There is no replication of any kind. This is a single instance server which
> alows certification login only.

Is is even available from the outside world? Else you should likely 
audit any internal hosts that could have accessed your postgresql 
server. If you have firewall logs looks for unusual connection attempts, 
any evidence of scanning, etc.

Hackers will often spend quite some time once inside to gather as much 
information as possible before doing any real damage, although if this 
is effectively a miner it would be less likely to be that kind of attack 
as they would probably not risk getting discovered with something that 
will at best make them pennies...

If you see the process having any open database files, it's possible 
it's either compressing them to exfiltrate the data or encrypting them, 
or both....

Hope this helps...

--
Thomas