LXC Out of memory - Containers using up memory!

Server Administration - Administration-du-Serveur
1

Hi All,

I am running a containerized environment, multiple DHIS2 containers (memory limit -16GB), the host is 32GB. Lately been experiencing out-of-memory issues, thus killing java related processes.

Error log is as below:

/var/log/kern.log:Jul 15 15:40:34 DHIS2devVM kernel: [ 2645.905748] oom_kill_process+0xe6/0x120
/var/log/kern.log:Jul 15 15:40:34 DHIS2devVM kernel: [ 2645.906045] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=postgres,mems_allowed=0,global_oom,task_memcg=/lxc/dev/system.slice/tomcat9.service,task=java,pid=28234,uid=100997
/var/log/kern.log:Jul 15 15:40:34 DHIS2devVM kernel: [ 2645.906114] Out of memory: Killed process 28234 (java) total-vm:5315692kB, anon-rss:631528kB, file-rss:0kB, shmem-rss:0kB, UID:100997 pgtables:1824kB oom_score_adj:0
Binary file /var/log/kern.log matches

As you can see, besides enforcing the memory limits and CPU cores for the dhis2 and Postgres containers, I have also set the Java heap size but still in vain, the host memory is being used up:
environment.JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
environment.JAVA_OPTS: ’ -Djava.awt.headless=true -XX:+UseG1GC -Xmx2G -Xms1G -Djava.security.egd=file:/dev/./urandom’

Running out of options on how to fix this, anyone know how to fix this? @bobj

Thanks,

2

Hi,

You can try using this options for java,

JAVA_OPTS=-XX:+UseG1GC -XX:+UseStringDeduplication -Xms512m -Xmx16384m -XX:-UseGCOverheadLimit -XX:MaxHeapFreeRatio=30 -XX:MinHeapFreeRatio=10

And tune pgoptions https://pgtune.leopard.in.ua.

4 Likes
3

Thanks, had to redeploy the containers, made sure to use this. Works really great. That pgtunes tools is amazing, thanks for sharing.

1 Like
4

@Mohammad_Ullah thank you for the support! @mykbitz, I’m so glad you got it solved :grin::+1:

1 Like