Performance issue for large number of enrollment event


I am working for the openIMIS initiative and I am building a Django module to push data to DHIS2 (program and dashboard were made by HISP india)
I am using DHIS2 2.35.1, 8BG ram allocated to Tomcat 8.5 and 2 GB allocated to PostgreSQL 11, hard drive is an SDD

I have two programs:
InsureeProgram with 3.2 millions enrollment to push in DHIS2 and then a bit more event (loading policy enrollment and renewal)
claimProgram with 1.6 million of enrollement (that include DE for the 3 stages)

When I started pushing I realized that it will take days sending insureeProgram enrollment 100 at the time (DHIS2 took 45 sec) so I decided to increase to 1000 but it was worst in term of perf (11 min) with the preheat mode “true” don’t change anything …
the wierd part is that 50 takes less than 11 sec

Is there a recommended size for the push ? what are the risk on pushing using different threads ?

Thanks in advance

FYI, I updated DHIS2 to 2.35.2 and it is much faster like 3sec for 100 enrollments, I will test with bigger batches

1 Like

@pmpdelcroix thanks for sharing!