Hi there DHIS2 devs,
We had an issue earlier where our DHIS2 server halted and we’re trying to track down what the issue was. It seems likely it was an issue of resources running out, most likely memory.
We currently have around 500 users but will eventually have something around 1200-1300 users. Each user is submitting data for a weekly aggregate report on a Monday morning, either via SMS or internet. They could submit any time between 6am - 4pm, with a heavy concentration coming in around 8-9am.
So a few questions then:
-
Does this seem reasonable?
-
What sort of memory requirements would you consider for this?
-
How many simultaneous submissions would be possible?
-
If it had more would the server queue them or would it cause it to halt?
Thanks for any help,
···
Jasper Timm
Lead Software Developer
eHealth Africa
4A Renner Drive
Off Wilkinson Road (behind Monoprix Supermarket)
Freetown, Sierra Leone
mobile +232 99891119
–
Jasper,
It’s not easy to get clear answers on your type of questions, from my experience, because there are so many variables…
1200-1300 users (i.e. each health facility in Sierra Leone) capturing weekly eIDSR data once per week + monthly capturing of a range of other data sets (currently captured at district level, but still per facility) should not be a problem at all - assuming, of course, that the configuration and resource allocation is reasonable. Remember here that there could have been some other heavy processing going on, including possibly some processes with memory leaks etc that temporarily causes the server to come to a halt. That type of “random” situation requiring a re-start of the web server to restore normal operations, I have seen a number of times - and tracking down whatever combinations of issues that triggered it is not always feasible, if it’s sort of a once-off affair.
But I will follow up your question more generally - the community need a more transparent & open discussion around scalability and performance, in particular because postgresql 10 and the just released version 11 in combination with recent DHIS2 versions offer several new options for table partitioning, load balancing etc. I believe there is very limited knowledge of and experience with these techniques in the wider DHIS2 community.
BTW: when you post questions like this, always specify whether the system is utilising any of the android apps (the old ones, at least, are more resource intensive than the browser interface) AND which DHIS2 version you are running (I know it is 2.27, but other don’t).
Best regards from Oslo
Calle
···
Calle Hedberg
46D Alma Road, 7700 Rosebank, SOUTH AFRICA
Tel/fax (home): +27-21-685-6472
Cell: +27-82-853-5352
Iridium SatPhone: +8816-315-19119
Email: calle.hedberg@gmail.com
Skype: calle_hedberg
Thanks for the response Calle.
I agree it’s often hard to track down resource issues like this. And yes it’d be great to get some documentation which would give a guide as to what sort of loads the server could run in terms of simultaneous submissions and browser sessions for a given CPU, RAM, disk spec e.t.c.
For more info on our case then - yes the DHIS2 version is 2.27. The mobile users submit via a tablet which will either submit to the DHIS2 API via internet or send an SMS, which will eventually make it’s way via a gateway to the DHIS2 API as well. Each of these submissions do not require a browser session at all, it should be one POST request to the DHIS2 API (either /dataValueSets for internet, or /sms/incoming for SMS). There’s usually no more than 1 submission every few seconds. That seems completely reasonable to me.
And the app is our modified version of the original DHIS2 data capture app.
···
Jasper Timm
Lead Software Developer
eHealth Africa
4A Renner Drive
Off Wilkinson Road (behind Monoprix Supermarket)
Freetown, Sierra Leone
mobile +232 99891119
–