Excessive Postgres Docker CPU Consumption

I'm using a Postgres container to run some small non-critical apps and sites. It's been stable for a while, but now the container has started to consume some serious CPU after it's been running for a short period of time. I have removed all other containers which use the Postgres container, and even after starting a new instance, the excessive CPU utilisation reoccurs. In my host (docker stats), I see this:

CONTAINER ID NAME
CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS cd553249727d data_postgresql.1.ft2gof5jci25xs5w5uqw6eezi
814.52% 22.11MiB / 46.95GiB 0.05% 129kB / 116kB 0B / 692kB 23


And this (top):
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
28923 70 20 0 633580 19664 488 S 696.7 0.0 2408:51 Dp2N


In the container (top), I see this:
Mem: 42042244K used, 7183656K free, 3622600K shrd, 1952K buff, 30585480K cached
CPU: 63% usr 9% sys 0% nic 26% idle 0% io 0% irq 0% sirq
Load average: 9.77 9.70 9.66 13/508 11090
PID PPID USER STAT VSZ %VSZ CPU %CPU COMMAND
94 1 postgres S 618m 1% 3 58% ./Dp2N

Query activity (no idea what select fun308928987('setsid ./systemd') does):
postgres=# select backend_start, usename, application_name, client_addr, client_hostname, query from pg_stat_activity;
backend_start | usename | application_name | client_addr | client_hostname | query
-------------------------------+------------+------------------+-------------+-----------------+-------------------------------------------------------------------------------------------------------------
2018-05-23 07:34:14.694057+00 | postgres | psql | | | select backend_start, usename, application_name, cl

You have been hacked, and are now mining cryptocurrency for the hacker.

They got in by guessing the password for your postgresql server's super-user account. Then they used the lo_export facility to drop the binary for a user-defined-function which executes arbitrary shell commands. That is what fun308928987 is, the SQL function which was created to wrap this binary.

Best clean up is to just destroy the server and rebuild it, this time setting up an actual strong password for the superuser account. Or better yet, also change pg_hba.conf to not allow super users connections, or preferably any connections, from the outside world.

StackExchange Database Administrators Q#207589, answer score: 8