OK! We must start from somewhere. Thank you!
I am not sure about thes calculations. And this is considering that we want to guarantee that Mr PG runs the query with all data on RAM. That to happen one needs special tunes for PG.
As a simple rule to guide us it is fine. And we need more benchmarks, simulations and more facts to evaluate variables.
···
On Thu, May 2, 2013 at 2:22 PM, Lars Helge Øverland larshelge@gmail.com wrote:
Hi Caveman,
thanks for the input. The memory recommendations are based on the size of the indexes which are created for analytics tables - ideally the database indexes on the analytics tables for the last 2 years should fit in memory for optimal performance. But this depends on the number of dynamic dimensions you have set up for your system - hence they should be considered a rule-of-thumb as I said. If you want to do testing specifically for your system then you can measure the size of the db indexes with SQL:
select pg_size_pretty(pg_total_relation_size(‘analytics_2012’));
minus this value
select pg_size_pretty(pg_relation_size(‘analytics_2012’));
Lars
On Thu, May 2, 2013 at 2:01 PM, Bob Jolliffe bobjolliffe@gmail.com wrote:
Hi Orvalho
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help : https://help.launchpad.net/ListHelp
On 1 May 2013 15:07, Orvalho Augusto orvaquim@gmail.com wrote:
It is true that hardware prices have decreased a lot.
But for me it seems too excessive these recommendations. So I am wondering how we arrived to these recommendations. Would be nice if we share that.
And for professional solutions it is not only one big machine you buy. You must have two. So when one dies you have the other ready.
Some other things to be tried are:
Instead of using one big muscular server you might need to consider to use one machine for webapplication and other database.
I agree that having a separate machine for the database can be a good thing. As long as they are connected with a good Gbs backbone. This also allows you to optimize the two machines differently. I know in Rwanda they have got away with fairly humble hardware so far by doing this. Though moving to 2.11 could still prove challenging.
Bob
And the webapplication server actually can be a group of machines doing it; And for databases you can have just one machine for writing queries which is master to other servers that are only queried for reading only (this for MySQL have been done) or other high availability solution.
For the tools to study your server, let me add some more:
- top - The basic to see the processes and CPU usage
- vmstat - This is verry important to see how your server is using the memory. Repeat it and check our the blocks are being used.
- plotting tools based on snmp may help to profile your server; To see correlations between traffic/connections and CPU or memory. So you can consider what to change.
Hope it helps.
Caveman
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help : https://help.launchpad.net/ListHelp
On Wed, May 1, 2013 at 12:21 PM, Dayo Adeyomoye deemoyes@yahoo.com wrote:
Yes you are right, hardware prices are coming down plus a very good server can run VMWare to properly allocate resources
BlackBerry: 3114F90C
Phone: 08028851441, 08035560463
emails: deemoyes@yahoo.com, addyraph@yahoo.com
Yahoo IM: deemoyes
Facebook: pharael
twitter:pharaell
From: Lars Helge Øverland larshelge@gmail.com
To: DHIS 2 developers dhis2-devs@lists.launchpad.net; "dhis2-users@lists.launchpad.net " dhis2-users@lists.launchpad.net
Sent: Wednesday, May 1, 2013 11:16 AM
Subject: [Dhis2-users] Server specifications
Hi all,
in version 2.11 and later DHIS performs aggregation on-the-fly as opposed to pre-aggregating data into data marts.
This raises the bar a little on your server’s specifications, mostly in terms of available RAM and disk read speeds.
Some rules-of-thumb server recommendations:
- RAM: At least 1 GB memory per 1 million captured data records per month or per 1000 concurrent users. At least 4 GB for a small instance, 12 GB for a medium instance.
- CPU cores: 4 CPU cores for a small instance, 8 CPU cores for a medium or large instance.
- Disk: Ideally use an SSD. Otherwise use a 7200 rpm disk. Minimum read speed is 150 Mb/s, 220 Mb/s is good, 350 Mb/s or better is ideal.
So we encourage you to invest in appropriate hardware to continue to benefit from all the new features in DHIS. Hardware prices are coming down and value for money is increasing at vps’ like linode and digitalocean.
regards,
Lars
PS. For server admins, here are some tips to measure hard disk/drive read speeds on Linux:
- Use the hdparm utility which will give you approximate read speed:
sudo hdparm -Tt /dev/sda
- If you are on a virtual server image without /dev/sda you can generate a random file and test write and read speed like this:
dd if=/dev/zero of=testfile bs=4k count=2000000
dd if=testfile of=/dev/null bs=4k count=2000000
- To do a real-life test, you can install iotop (sudo apt-get install iotop) and run it with
sudo iotop
then start DHIS in a new terminal and load a very large pivot table, then monitor read speed from iotop. You should be seeing read speeds around 12 - 25 Mb/s. Be aware that database queries will be cached after some time.
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help : https://help.launchpad.net/ListHelp