Removing the limit on the number of users to send a message

Hi Lars/Morten,

We want to remove the limit on the number of users to send a DHIS2(2.23) message. At the moment the limit is 200 users per group.

I would like to implement this change since we have a system that need to send 10 000+ emails to our users for notifications.

Was the limitation created to resolve server health issues when sending bulk emails?

Regards,

Mike

Hello Lars and team - thanks for having a look at this for us.

The requirement has emerged out of the success of the system Mike built to manage placement allocations for all Interns and Community Service Doctors in South Africa. Newly qualified doctors apply centrally, expressing their priority choices for hospital placements for their two years of internship and one year of community service. We run an algorithm to determine which hospital each doctor gets sent to.

Both doctors and government love the system. It’s email tool now provides the formal mechanism for communicating with these doctors about the placement process. Unfortunately, with the batch size limits, we’re having to setup the small batches manually. The current cohort of doctors is about 4,000 and the next cohort of nurses could reach 10,000.

Please advise if this needs to go into Launchpad or GitHub, or if you deal with this sort of request differently.

Your assistance will be very helpful.

Best regards,

Sean

This message and any attachments are subject to a disclaimer published at http://www.hisp.org/policies.html#comms_disclaimer. Please read the disclaimer before opening any attachment or taking any other action in terms of this electronic transmission. If you cannot access the disclaimer, kindly send an email to disclaimer@hisp.org and a copy will be provided to you. By replying to this e-mail or opening any attachment you agree to be bound by the provisions of the disclaimer.

···

On Mon, Oct 24, 2016 at 8:03 AM, Mike Nelushi mikeevolution@gmail.com wrote:

Hi Lars/Morten,

We want to remove the limit on the number of users to send a DHIS2(2.23) message. At the moment the limit is 200 users per group.

I would like to implement this change since we have a system that need to send 10 000+ emails to our users for notifications.

Was the limitation created to resolve server health issues when sending bulk emails?

Regards,

Mike

Dr Sean Broomhead

Director ICT

HISP

+27 79 496 1993

Skype: getsean

Hi,

Just to add one piece of info here: Having the ability to email or whatsapp more than 200-250 users is going to be critical for disease surveillance systems too. In case of a major disease outbreak, these systems will have to broadcast messages via multiple protocols to far more than 200-250 people.

Regards

Calle

···

On 24 October 2016 at 08:29, Sean Broomhead sean@hisp.org wrote:

Hello Lars and team - thanks for having a look at this for us.

The requirement has emerged out of the success of the system Mike built to manage placement allocations for all Interns and Community Service Doctors in South Africa. Newly qualified doctors apply centrally, expressing their priority choices for hospital placements for their two years of internship and one year of community service. We run an algorithm to determine which hospital each doctor gets sent to.

Both doctors and government love the system. It’s email tool now provides the formal mechanism for communicating with these doctors about the placement process. Unfortunately, with the batch size limits, we’re having to setup the small batches manually. The current cohort of doctors is about 4,000 and the next cohort of nurses could reach 10,000.

Please advise if this needs to go into Launchpad or GitHub, or if you deal with this sort of request differently.

Your assistance will be very helpful.

Best regards,

Sean

This message and any attachments are subject to a disclaimer published at http://www.hisp.org/policies.html#comms_disclaimer. Please read the disclaimer before opening any attachment or taking any other action in terms of this electronic transmission. If you cannot access the disclaimer, kindly send an email to disclaimer@hisp.org and a copy will be provided to you. By replying to this e-mail or opening any attachment you agree to be bound by the provisions of the disclaimer.

On Mon, Oct 24, 2016 at 8:03 AM, Mike Nelushi mikeevolution@gmail.com wrote:

Hi Lars/Morten,

We want to remove the limit on the number of users to send a DHIS2(2.23) message. At the moment the limit is 200 users per group.

I would like to implement this change since we have a system that need to send 10 000+ emails to our users for notifications.

Was the limitation created to resolve server health issues when sending bulk emails?

Regards,

Mike

Dr Sean Broomhead

Director ICT

HISP

+27 79 496 1993

Skype: getsean


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg


Hi there,

yes we can remove this for 2.26.

regards,

Lars

···

On Mon, Oct 24, 2016 at 4:02 AM, Calle Hedberg calle.hedberg@gmail.com wrote:

Hi,

Just to add one piece of info here: Having the ability to email or whatsapp more than 200-250 users is going to be critical for disease surveillance systems too. In case of a major disease outbreak, these systems will have to broadcast messages via multiple protocols to far more than 200-250 people.

Regards

Calle

On 24 October 2016 at 08:29, Sean Broomhead sean@hisp.org wrote:

Hello Lars and team - thanks for having a look at this for us.

The requirement has emerged out of the success of the system Mike built to manage placement allocations for all Interns and Community Service Doctors in South Africa. Newly qualified doctors apply centrally, expressing their priority choices for hospital placements for their two years of internship and one year of community service. We run an algorithm to determine which hospital each doctor gets sent to.

Both doctors and government love the system. It’s email tool now provides the formal mechanism for communicating with these doctors about the placement process. Unfortunately, with the batch size limits, we’re having to setup the small batches manually. The current cohort of doctors is about 4,000 and the next cohort of nurses could reach 10,000.

Please advise if this needs to go into Launchpad or GitHub, or if you deal with this sort of request differently.

Your assistance will be very helpful.

Best regards,

Sean

This message and any attachments are subject to a disclaimer published at http://www.hisp.org/policies.html#comms_disclaimer. Please read the disclaimer before opening any attachment or taking any other action in terms of this electronic transmission. If you cannot access the disclaimer, kindly send an email to disclaimer@hisp.org and a copy will be provided to you. By replying to this e-mail or opening any attachment you agree to be bound by the provisions of the disclaimer.


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg


On Mon, Oct 24, 2016 at 8:03 AM, Mike Nelushi mikeevolution@gmail.com wrote:

Hi Lars/Morten,

We want to remove the limit on the number of users to send a DHIS2(2.23) message. At the moment the limit is 200 users per group.

I would like to implement this change since we have a system that need to send 10 000+ emails to our users for notifications.

Was the limitation created to resolve server health issues when sending bulk emails?

Regards,

Mike

Dr Sean Broomhead

Director ICT

HISP

+27 79 496 1993

Skype: getsean

Lars Helge Øverland

Lead developer, DHIS 2

University of Oslo

Skype: larshelgeoverland

lars@dhis2.org

http://www.dhis2.org

Morning Lars - thank you. Do you need any further action from our side on this?

This message and any attachments are subject to a disclaimer published at http://www.hisp.org/policies.html#comms_disclaimer. Please read the disclaimer before opening any attachment or taking any other action in terms of this electronic transmission. If you cannot access the disclaimer, kindly send an email to disclaimer@hisp.org and a copy will be provided to you. By replying to this e-mail or opening any attachment you agree to be bound by the provisions of the disclaimer.

···

On Tue, Oct 25, 2016 at 3:54 AM, Lars Helge Øverland lars@dhis2.org wrote:

Hi there,

yes we can remove this for 2.26.

regards,

Lars

On Mon, Oct 24, 2016 at 4:02 AM, Calle Hedberg calle.hedberg@gmail.com wrote:

Hi,

Just to add one piece of info here: Having the ability to email or whatsapp more than 200-250 users is going to be critical for disease surveillance systems too. In case of a major disease outbreak, these systems will have to broadcast messages via multiple protocols to far more than 200-250 people.

Regards

Calle

Lars Helge Øverland

Lead developer, DHIS 2

University of Oslo

Skype: larshelgeoverland

lars@dhis2.org

http://www.dhis2.org

On 24 October 2016 at 08:29, Sean Broomhead sean@hisp.org wrote:

Hello Lars and team - thanks for having a look at this for us.

The requirement has emerged out of the success of the system Mike built to manage placement allocations for all Interns and Community Service Doctors in South Africa. Newly qualified doctors apply centrally, expressing their priority choices for hospital placements for their two years of internship and one year of community service. We run an algorithm to determine which hospital each doctor gets sent to.

Both doctors and government love the system. It’s email tool now provides the formal mechanism for communicating with these doctors about the placement process. Unfortunately, with the batch size limits, we’re having to setup the small batches manually. The current cohort of doctors is about 4,000 and the next cohort of nurses could reach 10,000.

Please advise if this needs to go into Launchpad or GitHub, or if you deal with this sort of request differently.

Your assistance will be very helpful.

Best regards,

Sean

This message and any attachments are subject to a disclaimer published at http://www.hisp.org/policies.html#comms_disclaimer. Please read the disclaimer before opening any attachment or taking any other action in terms of this electronic transmission. If you cannot access the disclaimer, kindly send an email to disclaimer@hisp.org and a copy will be provided to you. By replying to this e-mail or opening any attachment you agree to be bound by the provisions of the disclaimer.


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg


On Mon, Oct 24, 2016 at 8:03 AM, Mike Nelushi mikeevolution@gmail.com wrote:

Hi Lars/Morten,

We want to remove the limit on the number of users to send a DHIS2(2.23) message. At the moment the limit is 200 users per group.

I would like to implement this change since we have a system that need to send 10 000+ emails to our users for notifications.

Was the limitation created to resolve server health issues when sending bulk emails?

Regards,

Mike

Dr Sean Broomhead

Director ICT

HISP

+27 79 496 1993

Skype: getsean

Dr Sean Broomhead

Director ICT

HISP

+27 79 496 1993

Skype: getsean

Hi Mike (and Calle)

When you say

send 10 000+ emails
Does that mean 10.000 personalized (distinct) emails in one go, or is it a generic email going out to 10.000 email addresses in one go?

···

On Mon, Oct 24, 2016 at 8:03 AM, Mike Nelushi mikeevolution@gmail.com wrote:

Hi Lars/Morten,

We want to remove the limit on the number of users to send a DHIS2(2.23) message. At the moment the limit is 200 users per group.

I would like to implement this change since we have a system that need to send 10 000+ emails to our users for notifications.

Was the limitation created to resolve server health issues when sending bulk emails?

Regards,

Mike


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Halvdan Hoem Grelland

Software developer, DHIS 2

University of Oslo

http://www.dhis2.org

Hi Halvdan,

We send a generic email going out to 10.000 email addresses in one go most of the time, but the main issue is the email limitation.

Regards,

Mike

···

On Tue, Oct 25, 2016 at 10:26 AM, Halvdan Hoem Grelland halvdan@dhis2.org wrote:

Hi Mike (and Calle)

When you say

send 10 000+ emails
Does that mean 10.000 personalized (distinct) emails in one go, or is it a generic email going out to 10.000 email addresses in one go?

On Mon, Oct 24, 2016 at 8:03 AM, Mike Nelushi mikeevolution@gmail.com wrote:

Hi Lars/Morten,

We want to remove the limit on the number of users to send a DHIS2(2.23) message. At the moment the limit is 200 users per group.

I would like to implement this change since we have a system that need to send 10 000+ emails to our users for notifications.

Was the limitation created to resolve server health issues when sending bulk emails?

Regards,

Mike


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Halvdan Hoem Grelland

Software developer, DHIS 2

University of Oslo

http://www.dhis2.org

Ok, I see.

The limitation is there for a reason. Although it might be very low right now, I suspect churning out 10k emails directly from dhis2 will cause issues. They would be sent as 10k single messages through the configured SMTP which will be slow and might cause issues downstream as well (think rouge SMTP blacklisting etc).

Did you consider using a third party bulk email service, triggered by DHIS2 somehow?

···

On Tue, Oct 25, 2016 at 11:23 AM, Mike Nelushi mikeevolution@gmail.com wrote:

Hi Halvdan,

We send a generic email going out to 10.000 email addresses in one go most of the time, but the main issue is the email limitation.

Regards,

Mike

On Tue, Oct 25, 2016 at 10:26 AM, Halvdan Hoem Grelland halvdan@dhis2.org wrote:

Hi Mike (and Calle)

When you say

send 10 000+ emails
Does that mean 10.000 personalized (distinct) emails in one go, or is it a generic email going out to 10.000 email addresses in one go?

On Mon, Oct 24, 2016 at 8:03 AM, Mike Nelushi mikeevolution@gmail.com wrote:

Hi Lars/Morten,

We want to remove the limit on the number of users to send a DHIS2(2.23) message. At the moment the limit is 200 users per group.

I would like to implement this change since we have a system that need to send 10 000+ emails to our users for notifications.

Was the limitation created to resolve server health issues when sending bulk emails?

Regards,

Mike


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Halvdan Hoem Grelland

Software developer, DHIS 2

University of Oslo

http://www.dhis2.org

Halvdan Hoem Grelland

Software developer, DHIS 2

University of Oslo

http://www.dhis2.org

Halvdan

I think you have a point here - it would make sense to have both options, actually:

  1. Remove the limitation, BUT

  2. With an option to “export” a list of email addresses and the message to a bulk email server when the number of email recipients are above a Systems Setting specified number.

The main challenge is likely to be security, in the sense that the Dept of Health will NOT be willing to use a commercial service for this since it would give such a service access to all the emails for specific groups like all health graduates or all health staff (which they might misuse or sell to other companies doing marketing).

So we need to identify either a piece of open source software that can be incorporated into DHIS2 or at least run in parallel with DHIS2 - both will take a list of emails and a message and then bulk mail them out. I see sourceforge have a bunch of FOSS email senders, but not sure which is best (if any). Could be a commercial variety too, of course - I would not expect high cost.

Regards
Calle

···

On 25 October 2016 at 11:33, Halvdan Hoem Grelland halvdan@dhis2.org wrote:

Ok, I see.

The limitation is there for a reason. Although it might be very low right now, I suspect churning out 10k emails directly from dhis2 will cause issues. They would be sent as 10k single messages through the configured SMTP which will be slow and might cause issues downstream as well (think rouge SMTP blacklisting etc).

Did you consider using a third party bulk email service, triggered by DHIS2 somehow?


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

On Tue, Oct 25, 2016 at 11:23 AM, Mike Nelushi mikeevolution@gmail.com wrote:

Hi Halvdan,

We send a generic email going out to 10.000 email addresses in one go most of the time, but the main issue is the email limitation.

Regards,

Mike


Halvdan Hoem Grelland

Software developer, DHIS 2

University of Oslo

http://www.dhis2.org

On Tue, Oct 25, 2016 at 10:26 AM, Halvdan Hoem Grelland halvdan@dhis2.org wrote:

Hi Mike (and Calle)

When you say

send 10 000+ emails
Does that mean 10.000 personalized (distinct) emails in one go, or is it a generic email going out to 10.000 email addresses in one go?

On Mon, Oct 24, 2016 at 8:03 AM, Mike Nelushi mikeevolution@gmail.com wrote:

Hi Lars/Morten,

We want to remove the limit on the number of users to send a DHIS2(2.23) message. At the moment the limit is 200 users per group.

I would like to implement this change since we have a system that need to send 10 000+ emails to our users for notifications.

Was the limitation created to resolve server health issues when sending bulk emails?

Regards,

Mike


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Halvdan Hoem Grelland

Software developer, DHIS 2

University of Oslo

http://www.dhis2.org


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg