Will be interested to see what comes back from your clustering experiments.
Regarding splitting server in two, in my experience if you have a separate server it can be good to dedicate that to postgresql. This allows the database server to be singularly tuned as a database server. Assumes you have guaranteed high bandwidth/low latency between database and application server(s).
···
On 23 September 2013 18:33, Ngoc Thanh Nguyen thanh.hispvietnam@gmail.com wrote:
ok, Jason. I have some quick note but will detail it and send back to you
Thanh
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help : https://help.launchpad.net/ListHelp
Nguyễn Ngọc Thành
On Mon, Sep 23, 2013 at 10:42 PM, Jason Pickering jason.p.pickering@gmail.com wrote:
Hi Thanh,
Lets put it back on the list with your findings.I wanted to be sure we had some consensus before proceeding too much with that line. I think the configuration steps you took would be great to have in the documents. Why don’t you summarize what you find out and i can help you write everything up into the docs as an alternative, which is perhaps better. The example I gave was complete overkill, but maybe you found a good balance between having a highly available cluster, versus a complete overkill (which I think was my problem).
Regards,
Jason
On Mon, Sep 23, 2013 at 5:28 PM, Ngoc Thanh Nguyen thanh.hispvietnam@gmail.com wrote:
Jason, thanks for the discussion. I eventually can get dhis2 work with Tomcat clustering. Persisting sessions in jdbc works like a charm with very less configuration. Aggregation data entry could happen quite rare, every month or every quarter. However, individual data capture can happen daily in real time so being down for 1 hour could be catastrophe. My argument is dhis2 can be even better if it is configured followed strict standards of commercial systems such as High Availability, High Scalability. Instead of having 1 server with 16G RAM, why don’t we have 2 separate ones with 8GB each?
I don’t know why our discussion goes off the list?
Thanh
Nguyễn Ngọc Thành
On Mon, Sep 23, 2013 at 10:37 AM, Jason Pickering jason.p.pickering@gmail.com wrote:
Of course it depends on the use case, my point here was what is the use case?? I just did not want you trying to prove to some corporate people that just because we are using an OpenSource product like DHIS2/Tomcat, that it is somehow less robust than some commercial equivalent, for the vast majority of use cases. But now it seems the requirement is different.
I think for the case you mention, that has never been tested or confirmed I think and would need you to implement session sharing across multiple tomcat instances. That sounds a lot more difficult to do than a simple cluster. Even if it did work, it would require users (usually or often) to clear their browser cache anyway. IMHO, Upgrades should something which should be done at night, with few users active. Normally, they should take less than two minutes, again depending on the size of your server. Upgrades to a production server should be tested in a testing environment and confirmed to work, as usually, they are things you cannot easily roll-.back. So, if you are upgrading a production instance every day or multiple times a day, it would seem to me you are asking for trouble. Again, just the way I feel about it, but trying to implement a clustering solution to solve this “requirement” sounds pretty risky to me.
Would certainly be worth to investigate and see if you can get it to work, as it would certainly help increase redundancy of the deployments.
Regards,
Jason
On Mon, Sep 23, 2013 at 1:48 AM, Ngoc Thanh Nguyen thanh.hispvietnam@gmail.com wrote:
It is, Jason. I think we will need a cluster in case of upgrading dhis2. When you want to change a jar file, you have to shutdown the whole application but if you have 2 servers you can do one by one and there is no pause in the service.
I think it depends on the use case.
Nguyễn Ngọc Thành
On Sun, Sep 22, 2013 at 9:47 PM, Jason Pickering jason.p.pickering@gmail.com wrote:
Hi Thanh,
Going to take this off the list for the moment, as it may not be so constructive.
I would take a step back and ask what the use case is. Why do you need a cluster? Do you know you need a cluster?
I think doing it could be a fun way to spend a Sunday afternoon, but my wife would probably think it better to spend time with the kids. My point is that unless you know you need to do it, don’t bother, because it has been done before, and it is nothing exotic. It may be good experience for you, so could be worth it. However, setting up a DB cluster, Tomcat cluster, foo cluster has been done by many others, and the process is well documented and proven, so I see no particular need to prove we can setup DHIS2 in this environment. We know that it can be done, because it has been done with others with very similar types of Java applications. I have setup a similar type of architecture with Amazon RDS in the background, fleets of application servers, fronted by load balancers (Apache). Nothing really so fancy. But it turned out to be complete overkill, and we went back to a single server with 16 GB of memory, and it works just fine for out needs.
Regards,
Jason
On Sun, Sep 22, 2013 at 3:45 PM, Ngoc Thanh Nguyen thanh.hispvietnam@gmail.com wrote:
Thanks Jason. The reply is insightful.
I now try to set up dhis2 on a Tomcat cluster with nginx and mysql cluster altogether. Will let you know the result soon
Nguyễn Ngọc Thành
On Sun, Sep 22, 2013 at 12:26 PM, Knut Staring knutst@gmail.com wrote:
I think we should get DHIS2 added to that list of systems (typically) running on Tomcat. It is a MoinMoin wiki, so I thought I could just add us, but it seems this is an “immutable” page…
On Sun, Sep 22, 2013 at 7:19 AM, Jason Pickering jason.p.pickering@gmail.com wrote:
Hi Thanh,
First off, be sure you are aware of who you are dealing with in such a debate. These “commercial companies” are probably a lot better at you in selling products to people who do not need them than you are. This is after all, what they do. So, keeping that in mind, lets pick apart what they are saying.
“DHIS2 is not built on and use well-know reliable solutions such as Microsoft Sharepoint, IBM Websphere, Oracle Bea Weblogic”
No reason why you could not used DHIS2 with Websphere or Weblogic in theory. Of course, it would be a huge waste of money to plonk down money for these applications, when Apache Tomcat (which is free) will do the job just fine. DHIS2 is using Spring and not EJB, so there is simply no need for these products. As for Sharepoint, well, it is a totally different application stack, and not really interesting to debate about. Point is DHIS2 is built using Java and lots of standard open-source libraries used by lots of other projects (including commercial ones). So which application server it uses is really a mater of choice. You could take Tomcat, and it would work fine. You could also pay a company a lot of money for a licence for one of the products you mention. Here is a partial list of companies/projects using Tomcat, and the number is not small (http://wiki.apache.org/tomcat/PoweredBy)
- When multiple DHIS2 instances can be deployed in multiple servers (virtual or physical) and load balanced by a hard or soft balancer (nginxs), this can not solve the real >problem of distributed HTTP sessions like what can be done in Application Server, i.e. a logged in users connected to a instance and this instance die, how can they continue >without relogin.
Can using Application Server solve the problems without having to rewrite dhis2?
Which Application Server? I guess my question here would be what is the impact of a clustered Tomcat instance crashing? OK, maybe the user has to login again. Maybe they loose a little bit of work at worst. So what? Not that this is not an impact, but would it justify even an attempt to implement a load-balanced clustered solution with some sort of failover protection transparent to the user? It would not really seem to be justified.
How are we going to solve this? Will we need to use commercial solutions like Terracotta BigMemory or IBM Data Grid?
What is the point? This again sounds like a bit of a snake-oil argument here. These products are not going to help with the issue of an application server failing. .
- I would like to know the typical deployment of dhis2 in other countries. Is that on one or multiple server, with or without database clusters? I heard that many countries go for >cloud but I am not sure if it is a single or multiple cloud servers?
Most countries which I know of are using a rather modest server. Many are using the cloud, some are not. Many countries have everything on the same machine (database, application server, nginx) and separating the database and application server is fairly typical as well. But I could imagine there are some installations out there which are using clusters, but I would say this is probably not typical (yet) of most DHIS2 installations.
Can cloud provide us a single virtual but unlimited RAM and CPUs server? As if yes, then we don’t have to worry about question 1, 2 because we can simple add 10000000 >GB RAM and 100000 CPUs to that cloud server.
Well, for all intents and purposes, you could continue to increase the size of the server up to a certain size as required but I think the metrics you give there are pretty ridiculous.Why would you need that much RAM or CPU power? Even if it were physically possible which seems reasonable (although likely very very expensive) , it would generally not make much sense to do this. Probably better to break the system up into smaller pieces instead. Using many small servers to distribute the process is typically the favored approach, instead of having one big massive machine.
Anyway, there are a few thoughts off the top of my head. In summary, there is simply no need for the commercial products you mention in the vast majority of use cases we have come across. Not to say that they do not exist, but I would press these “companies” for more details on how these are actual requirements, and not simply invented hypothetical cases which could potentially be used to make a sale of unneeded licences.
Regards,
Jason
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help : https://help.launchpad.net/ListHelp
Knut Staring
Dept. of Informatics, University of Oslo
+4791880522
http://dhis2.org
–
On Sun, Sep 22, 2013 at 3:14 AM, Ngoc Thanh Nguyen thanh.hispvietnam@gmail.com wrote:
Hi all,
Some commercial affiliated companies want to kick dhis2 out so they made the following arguments: “DHIS2 is not built on and use well-know reliable solutions such as Microsoft Sharepoint, IBM Websphere, Oracle Bea Weblogic”
When most of this is wrong they have some valid points that we need to be prepared in order to fight back. I would like to get your ideas on
- When multiple DHIS2 instances can be deployed in multiple servers (virtual or physical) and load balanced by a hard or soft balancer (nginxs), this can not solve the real problem of distributed HTTP sessions like what can be done in Application Server, i.e. a logged in users connected to a instance and this instance die, how can they continue without relogin.
Can using Application Server solve the problems without having to rewrite dhis2?
How are we going to solve this? Will we need to use commercial solutions like Terracotta BigMemory or IBM Data Grid?
- For database scale, with one database instances installed on single server. it has the risk of data loss. We can use database clusters to address this issue, and data on clusters mostly rely on memory. My question is how this cluster will work with distributed sessions described in question 1.
- I would like to know the typical deployment of dhis2 in other countries. Is that on one or multiple server, with or without database clusters? I heard that many countries go for cloud but I am not sure if it is a single or multiple cloud servers?
Can cloud provide us a single virtual but unlimited RAM and CPUs server? As if yes, then we don’t have to worry about question 1, 2 because we can simple add 10000000 GB RAM and 100000 CPUs to that cloud server.
Thanks in advanced.
Thanh
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help : https://help.launchpad.net/ListHelp