For the past two days I’ve woken up to this after logging in to DHIS2 (2.16) on Windows:
HTTP Status 500 - Hazelcast instance is not active!
I’m not sure what’s causing it and I need to investigate the logs but I wondered if anything obvious comes to mind from the user group – has anyone else experienced this and what should I look out for?
I’ll reboot the server (sorts it out) and continue with some urgent work then investigate what may be causing this.
Thanks!
Ed
For reference, this is my setup according to the ‘about DHIS2’ page:
From: Dhis2-users [mailto:dhis2-users-bounces+erobinson=projectbalance.com@lists.launchpad.net] On Behalf Of Edward Robinson
Sent: Friday, 28 July 2017 11:48 AM
To: dhis2-users dhis2-users@lists.launchpad.net
Subject: [Dhis2-users] Hazelcast instance not active
For the past two days I’ve woken up to this after logging in to DHIS2 (2.16) on Windows:
HTTP Status 500 - Hazelcast instance is not active!
I’m not sure what’s causing it and I need to investigate the logs but I wondered if anything obvious comes to mind from the user group – has anyone else experienced this and what should I look out for?
I’ll reboot the server (sorts it out) and continue with some urgent work then investigate what may be causing this.
Thanks!
Ed
Hi Edward,
You would be well advised to upgrade that instance as soon as possible. There are number of very serious security issues which have been fixed in later versions, but not as far as I know, as far back as 2.16. It would not surprise me at all of your server had been compromised, which might explain why this is happening. I would carefully check the server logs for any strange activity, but would recommend that you upgrade to a later version as soon as possible, where a number of security problems have been recently rectified.
Regards,
Jason
On Fri, Jul 28, 2017 at 11:55 AM, Edward Robinson erobinson@projectbalance.com wrote:
For reference, this is my setup according to the ‘about DHIS2’ page:
From: Dhis2-users [mailto:dhis2-users-bounces+erobinson=projectbalance.com@lists.launchpad.net] On Behalf Of Edward Robinson
Sent: Friday, 28 July 2017 11:48 AM
To: dhis2-users dhis2-users@lists.launchpad.net
Subject: [Dhis2-users] Hazelcast instance not active
For the past two days I’ve woken up to this after logging in to DHIS2 (2.16) on Windows:
HTTP Status 500 - Hazelcast instance is not active!
I’m not sure what’s causing it and I need to investigate the logs but I wondered if anything obvious comes to mind from the user group – has anyone else experienced this and what should I look out for?
I’ll reboot the server (sorts it out) and continue with some urgent work then investigate what may be causing this.
Thanks!
Ed
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp
–
Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+46764147049
Hi Jason, indeed, upgrading the instance is very high on the priority list. This is a server that we have taken over. It had been compromised previously on Linux with a well known bitcoin mining zero-day vulnerability affecting out of date struts instances. That has since been sorted out. A second server running Windows has not been affected. We had migrated this machine to a Windows box as a short term solution since the zero-day script was written to specifically target Linux. Since Upgrading is a time consuming process, we are scheduling it for ‘as soon as possible’ beginning this weekend. It will happen off-line, incrementally, until we are up to date but it’s not likely to be complete in the next few days.
For now, is there anything obviously amiss in the reported output from ‘about DHIS2’?
Thanks!
Ed
Hi Edward,
You would be well advised to upgrade that instance as soon as possible. There are number of very serious security issues which have been fixed in later versions, but not as far as I know, as far back as 2.16. It would not surprise me at all of your server had been compromised, which might explain why this is happening. I would carefully check the server logs for any strange activity, but would recommend that you upgrade to a later version as soon as possible, where a number of security problems have been recently rectified.
Regards,
Jason
On Fri, Jul 28, 2017 at 11:55 AM, Edward Robinson erobinson@projectbalance.com wrote:
For reference, this is my setup according to the ‘about DHIS2’ page:
From: Dhis2-users [mailto:dhis2-users-bounces+erobinson=projectbalance.com@lists.launchpad.net] On Behalf Of Edward Robinson
Sent: Friday, 28 July 2017 11:48 AM
To: dhis2-users dhis2-users@lists.launchpad.net
Subject: [Dhis2-users] Hazelcast instance not active
For the past two days I’ve woken up to this after logging in to DHIS2 (2.16) on Windows:
HTTP Status 500 - Hazelcast instance is not active!
I’m not sure what’s causing it and I need to investigate the logs but I wondered if anything obvious comes to mind from the user group – has anyone else experienced this and what should I look out for?
I’ll reboot the server (sorts it out) and continue with some urgent work then investigate what may be causing this.
Thanks!
Ed
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp
–
Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+46764147049
Hi Edward,
The security issues I am referring to are related to vulnerabilities in component libraries of DHIS2, as you note. They have nothing to due with the underlying operating system itself, so even if you move to Windows, you will not be any safer, as the vulnerabilities exist in the DHIS2 software itself.
Perhaps moving your server prevented the attack from happening again?
Regardless, Hazelcast is no longer in use, so I think once you upgrade, that problem should disappear.
Regards,
Jason
On Fri, Jul 28, 2017 at 12:16 PM, Edward Robinson erobinson@projectbalance.com wrote:
Hi Jason, indeed, upgrading the instance is very high on the priority list. This is a server that we have taken over. It had been compromised previously on Linux with a well known bitcoin mining zero-day vulnerability affecting out of date struts instances. That has since been sorted out. A second server running Windows has not been affected. We had migrated this machine to a Windows box as a short term solution since the zero-day script was written to specifically target Linux. Since Upgrading is a time consuming process, we are scheduling it for ‘as soon as possible’ beginning this weekend. It will happen off-line, incrementally, until we are up to date but it’s not likely to be complete in the next few days.
For now, is there anything obviously amiss in the reported output from ‘about DHIS2’?
Thanks!
Ed
From: Jason Pickering [mailto:jason.p.pickering@gmail.com]
Sent: Friday, 28 July 2017 11:58 AM
To: Edward Robinson erobinson@projectbalance.com
Cc: dhis2-users dhis2-users@lists.launchpad.net
Subject: Re: [Dhis2-users] Hazelcast instance not active
Hi Edward,
You would be well advised to upgrade that instance as soon as possible. There are number of very serious security issues which have been fixed in later versions, but not as far as I know, as far back as 2.16. It would not surprise me at all of your server had been compromised, which might explain why this is happening. I would carefully check the server logs for any strange activity, but would recommend that you upgrade to a later version as soon as possible, where a number of security problems have been recently rectified.
Regards,
Jason
On Fri, Jul 28, 2017 at 11:55 AM, Edward Robinson erobinson@projectbalance.com wrote:
For reference, this is my setup according to the ‘about DHIS2’ page:
From: Dhis2-users [mailto:dhis2-users-bounces+erobinson=projectbalance.com@lists.launchpad.net] On Behalf Of Edward Robinson
Sent: Friday, 28 July 2017 11:48 AM
To: dhis2-users dhis2-users@lists.launchpad.net
Subject: [Dhis2-users] Hazelcast instance not active
For the past two days I’ve woken up to this after logging in to DHIS2 (2.16) on Windows:
HTTP Status 500 - Hazelcast instance is not active!
I’m not sure what’s causing it and I need to investigate the logs but I wondered if anything obvious comes to mind from the user group – has anyone else experienced this and what should I look out for?
I’ll reboot the server (sorts it out) and continue with some urgent work then investigate what may be causing this.
Thanks!
Ed
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp
–
Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+46764147049
–
Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+46764147049
Great, thanks Jason – yes, the particular rootkit that was used on the previous Linux instance was written specifically for Linux but of course that’s not an assumption that a Windows box won’t be affected by another vulnerability, so we are certainly taking it seriously. Great to know the Hazelcast issue at least will be resolved.
Is there a minimum recommended version we should be upgrading to from a security standpoint? I only ask since I’m not under any illusions about it being a seamless and smooth process with so many versions to get through and in case we have trouble along the way, I just wondered if we could say, aim for a minimum of version 2.xx as urgent with anything above that for now being bonus.
Lastly, I had trouble finding older versions of the software online and since I’m likely going to do this one version at a time, is there somewhere I can find a list of instructions on moving from one version to the next starting at 2.16 as well as the .war files?
Regards
Ed
Hi Edward,
The security issues I am referring to are related to vulnerabilities in component libraries of DHIS2, as you note. They have nothing to due with the underlying operating system itself, so even if you move to Windows, you will not be any safer, as the vulnerabilities exist in the DHIS2 software itself.
Perhaps moving your server prevented the attack from happening again?
Regardless, Hazelcast is no longer in use, so I think once you upgrade, that problem should disappear.
Regards,
Jason
On Fri, Jul 28, 2017 at 12:16 PM, Edward Robinson erobinson@projectbalance.com wrote:
Hi Jason, indeed, upgrading the instance is very high on the priority list. This is a server that we have taken over. It had been compromised previously on Linux with a well known bitcoin mining zero-day vulnerability affecting out of date struts instances. That has since been sorted out. A second server running Windows has not been affected. We had migrated this machine to a Windows box as a short term solution since the zero-day script was written to specifically target Linux. Since Upgrading is a time consuming process, we are scheduling it for ‘as soon as possible’ beginning this weekend. It will happen off-line, incrementally, until we are up to date but it’s not likely to be complete in the next few days.
For now, is there anything obviously amiss in the reported output from ‘about DHIS2’?
Thanks!
Ed
From: Jason Pickering [mailto:jason.p.pickering@gmail.com]
Sent: Friday, 28 July 2017 11:58 AM
To: Edward Robinson erobinson@projectbalance.com
Cc: dhis2-users dhis2-users@lists.launchpad.net
Subject: Re: [Dhis2-users] Hazelcast instance not active
Hi Edward,
You would be well advised to upgrade that instance as soon as possible. There are number of very serious security issues which have been fixed in later versions, but not as far as I know, as far back as 2.16. It would not surprise me at all of your server had been compromised, which might explain why this is happening. I would carefully check the server logs for any strange activity, but would recommend that you upgrade to a later version as soon as possible, where a number of security problems have been recently rectified.
Regards,
Jason
On Fri, Jul 28, 2017 at 11:55 AM, Edward Robinson erobinson@projectbalance.com wrote:
For reference, this is my setup according to the ‘about DHIS2’ page:
From: Dhis2-users [mailto:dhis2-users-bounces+erobinson=projectbalance.com@lists.launchpad.net] On Behalf Of Edward Robinson
Sent: Friday, 28 July 2017 11:48 AM
To: dhis2-users dhis2-users@lists.launchpad.net
Subject: [Dhis2-users] Hazelcast instance not active
For the past two days I’ve woken up to this after logging in to DHIS2 (2.16) on Windows:
HTTP Status 500 - Hazelcast instance is not active!
I’m not sure what’s causing it and I need to investigate the logs but I wondered if anything obvious comes to mind from the user group – has anyone else experienced this and what should I look out for?
I’ll reboot the server (sorts it out) and continue with some urgent work then investigate what may be causing this.
Thanks!
Ed
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp
–
Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+46764147049
–
Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+46764147049
