Reports - where's the data?

Ok, I’m feeling quite stupid here.
We’ve been entering monthly data for 2011 with a form and we can view this using the “Dataset” report and it aggregates for province and for quarters and years.
Now, I want to put together some reports.

I try the Standard report… this requires a Table.
I put together a Table report and run it… nice table with data elements, org units and periods… but no data! (blank, nada).
I look at the web Pivot tables and run a few… nice pivot tables, rows, columns, periods… no data!.. blanks… not even a single number anywhere!

What am I missing here?

.Mark

···


Mark Spohr, MD
mhspohr@gmail.com
+1 530 554 2230

Hi mark, try to run the datamart, jørn

···

On 3/20/12, Mark Spohr <mhspohr@gmail.com> wrote:

Ok, I'm feeling quite stupid here.
We've been entering monthly data for 2011 with a form and we can view this
using the "Dataset" report and it aggregates for province and for quarters
and years.
Now, I want to put together some reports.
I try the Standard report... this requires a Table.
I put together a Table report and run it... nice table with data elements,
org units and periods... but no data! (blank, nada).
I look at the web Pivot tables and run a few... nice pivot tables, rows,
columns, periods... no data!... blanks... not even a single number anywhere!

What am I missing here?

.Mark

--
Mark Spohr, MD
mhspohr@gmail.com
+1 530 554 2230

--
Sent from my mobile device

Report data will generate from entered data by nightly batch job, usually.

You must go to data administration module, then “Scheduling”. It should be on automatically. But it might run out of memory as it uses a lot of resources.

Please try this.

         Suzuki
···

From: dhis2-users-bounces+shin461=gmail.com@lists.launchpad.net [mailto:dhis2-users-bounces+shin461=gmail.com@lists.launchpad.net] On Behalf Of Mark Spohr
Sent: Tuesday, March 20, 2012 8:00 AM
To: dhis2-users@lists.launchpad.net
Subject: [Dhis2-users] Reports - where’s the data?

Ok, I’m feeling quite stupid here.
We’ve been entering monthly data for 2011 with a form and we can view this using the “Dataset” report and it aggregates for province and for quarters and years.
Now, I want to put together some reports.
I try the Standard report… this requires a Table.
I put together a Table report and run it… nice table with data elements, org units and periods… but no data! (blank, nada).
I look at the web Pivot tables and run a few… nice pivot tables, rows, columns, periods… no data!.. blanks… not even a single number anywhere!

What am I missing here?

.Mark


Mark Spohr, MD
mhspohr@gmail.com
+1 530 554 2230

HI Mark,

(copying dev list also since many people ask about this)

Here are datamart instructions for version 2.7:

Automatic datamart:

(2.6 and 2.7)

For servers we recommend you use the automatic scheduled datamart (runs at midnight every day).

This can be configured in Data Administration-Scheduling. (see user manual section 19.19)

It is correctly set up when it says “Scheduling is running” - which means that DHIS is scheduled to run a datamart export the next (and every) midnight. No need to click on Execute now (that starts the datamart process immediately, but this is sometimes needed as well)

Note that this scheduled datamart will export maximum data for 12 months back in time, (but will not delete older data). If you have an empty datamart and have data older than 12 months you should use the manual approach (see below) for those months that are older than 1 year.

How fast this scheduled datamart export process is will really depend on your CPU and Memory, but it will most likely take a while. In the tomcat log you can follow progress and CPU and memory use etc.

Manual datamart:

(this is or 2.7)

If you need to do manual datamart exports these are now possible from Reports->Datamart, but this is more simple now, it takes all data within the period you specify. This has been made much easier now.

(see section 13.2.1 in the user manual, it has been updated to this new datamart window).

(2.6)

A bit more complicated UI for datamart export accessed under Services->Datamart

Here you can select exactly what you want to export. Note that this export (2.6 and earlier) does not rebuild indexes on the datamart tables.

After an export, go to Maintenance->Data Administration->Maintenance: and tick Rebuild datamart indexes. Without that last step the reports will be a lot slower, especially on a multi-user environment.

Ola

···

Ola Hodne Titlestad (Mr)
HISP
Department of Informatics
University of Oslo

Mobile: +47 48069736
Home address: Vetlandsvn. 95B, 0685 Oslo, Norway. Googlemaps link

On 20 March 2012 09:49, Shinichi Suzuki shin461@gmail.com wrote:

Report data will generate from entered data by nightly batch job, usually.

You must go to data administration module, then “Scheduling”. It should be on automatically. But it might run out of memory as it uses a lot of resources.

Please try this.

         Suzuki

From: dhis2-users-bounces+shin461=gmail.com@lists.launchpad.net [mailto:dhis2-users-bounces+shin461=gmail.com@lists.launchpad.net] On Behalf Of Mark Spohr

Sent: Tuesday, March 20, 2012 8:00 AM
To: dhis2-users@lists.launchpad.net

Subject: [Dhis2-users] Reports - where’s the data?

Ok, I’m feeling quite stupid here.
We’ve been entering monthly data for 2011 with a form and we can view this using the “Dataset” report and it aggregates for province and for quarters and years.

Now, I want to put together some reports.
I try the Standard report… this requires a Table.
I put together a Table report and run it… nice table with data elements, org units and periods… but no data! (blank, nada).

I look at the web Pivot tables and run a few… nice pivot tables, rows, columns, periods… no data!.. blanks… not even a single number anywhere!

What am I missing here?

.Mark

Mark Spohr, MD
mhspohr@gmail.com
+1 530 554 2230


Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp

We’ve got data!!

Thanks everyone for your help. I was confused and thought the Datamart utility was for the external program MyDatamart. I ran it and now my reports have actual data in them. This is a big improvement. I’ve set up scheduling to keep it up to date.

I also love the Data Visualizer… quite posh!

Thanks again,
Mark

···

On Tue, Mar 20, 2012 at 3:02 AM, Ola Hodne Titlestad olati@ifi.uio.no wrote:

HI Mark,

(copying dev list also since many people ask about this)

Here are datamart instructions for version 2.7:

Automatic datamart:

(2.6 and 2.7)

For servers we recommend you use the automatic scheduled datamart (runs at midnight every day).

This can be configured in Data Administration-Scheduling. (see user manual section 19.19)

It is correctly set up when it says “Scheduling is running” - which means that DHIS is scheduled to run a datamart export the next (and every) midnight. No need to click on Execute now (that starts the datamart process immediately, but this is sometimes needed as well)

Note that this scheduled datamart will export maximum data for 12 months back in time, (but will not delete older data). If you have an empty datamart and have data older than 12 months you should use the manual approach (see below) for those months that are older than 1 year.

How fast this scheduled datamart export process is will really depend on your CPU and Memory, but it will most likely take a while. In the tomcat log you can follow progress and CPU and memory use etc.

Manual datamart:

(this is or 2.7)

If you need to do manual datamart exports these are now possible from Reports->Datamart, but this is more simple now, it takes all data within the period you specify. This has been made much easier now.

(see section 13.2.1 in the user manual, it has been updated to this new datamart window).

(2.6)

A bit more complicated UI for datamart export accessed under Services->Datamart

Here you can select exactly what you want to export. Note that this export (2.6 and earlier) does not rebuild indexes on the datamart tables.

After an export, go to Maintenance->Data Administration->Maintenance: and tick Rebuild datamart indexes. Without that last step the reports will be a lot slower, especially on a multi-user environment.

Ola



Ola Hodne Titlestad (Mr)
HISP
Department of Informatics
University of Oslo

Mobile: +47 48069736
Home address: Vetlandsvn. 95B, 0685 Oslo, Norway. Googlemaps link

On 20 March 2012 09:49, Shinichi Suzuki shin461@gmail.com wrote:

Report data will generate from entered data by nightly batch job, usually.

You must go to data administration module, then “Scheduling”. It should be on automatically. But it might run out of memory as it uses a lot of resources.

Please try this.

         Suzuki

From: dhis2-users-bounces+shin461=gmail.com@lists.launchpad.net [mailto:dhis2-users-bounces+shin461=gmail.com@lists.launchpad.net] On Behalf Of Mark Spohr

Sent: Tuesday, March 20, 2012 8:00 AM
To: dhis2-users@lists.launchpad.net

Subject: [Dhis2-users] Reports - where’s the data?

Ok, I’m feeling quite stupid here.
We’ve been entering monthly data for 2011 with a form and we can view this using the “Dataset” report and it aggregates for province and for quarters and years.

Now, I want to put together some reports.
I try the Standard report… this requires a Table.
I put together a Table report and run it… nice table with data elements, org units and periods… but no data! (blank, nada).

I look at the web Pivot tables and run a few… nice pivot tables, rows, columns, periods… no data!.. blanks… not even a single number anywhere!

What am I missing here?

.Mark

Mark Spohr, MD
mhspohr@gmail.com
+1 530 554 2230


Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp


Mark Spohr, MD
mhspohr@gmail.com
+1 530 554 2230

Hi all,

The Rwanda team now has 3 months of data from over 400 sites in the DHIS-2. We’re impressed with how well it has worked even in low bandwidth conditions. Congratulations to the team who built in the offline editing functionality.

Now we’re on to reporting and find a number of immediate concerns:

  1.   Data completeness report: There does not seem to be a report that lists the orgunits that did not report for a given period for a given dataset.  We need this to issue reminders to data managers and their district supervisors so that data entry is complete.
    
  2.   When we create a report-table for any data elements in a data set and list leaf orgunits, it lists all orgunits at that level regardless of whether or not they are supposed to report for that dataset.  In Rwanda we have health facilities and a number of administrative units (Health Insurance offices) at the same orgunit level and we can’t filter out the latter.  There also does not appear to be a way to include frequently used orgunitgroups like type (health center, hospital, referral hospital, etc…) or ownership in the filter criteria for these report tables.
    

Any suggestions would be very helpful.

Randy Wilson

Hi Randy,

How about using a custom report? There is a quite good section in the
implementers manual on how to use iReport.

Best regards,
Jason

···

On Wed, Mar 21, 2012 at 9:14 AM, Wilson,Randy <rwilson@msh.org> wrote:

Hi all,

The Rwanda team now has 3 months of data from over 400 sites in the DHIS-2.
We’re impressed with how well it has worked even in low bandwidth
conditions. Congratulations to the team who built in the offline editing
functionality.

Now we’re on to reporting and find a number of immediate concerns:

1. Data completeness report: There does not seem to be a report that
lists the orgunits that did not report for a given period for a given
dataset. We need this to issue reminders to data managers and their
district supervisors so that data entry is complete.

2. When we create a report-table for any data elements in a data set
and list leaf orgunits, it lists all orgunits at that level regardless of
whether or not they are supposed to report for that dataset. In Rwanda we
have health facilities and a number of administrative units (Health
Insurance offices) at the same orgunit level and we can’t filter out the
latter. There also does not appear to be a way to include frequently used
orgunitgroups like type (health center, hospital, referral hospital, etc…)
or ownership in the filter criteria for these report tables.

Any suggestions would be very helpful.

Randy Wilson

_______________________________________________
Mailing list: DHIS 2 Users in Launchpad
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : DHIS 2 Users in Launchpad
More help : ListHelp - Launchpad Help

Hi Randy, there's no in-built report for this, but it should be possible to build a custom report as Jason suggests. However, you can find the orgunits if you want, if you drill down in the Reporting Rate Summary page of Reports.

Example from online demo (Sierra Leone orgunits):
I select top orgunit (Sierra Leone), and the dataset and period. I then get the number of reporting units for each of the next level (district) that should report (target) and those that actually have reported (actual).

For instance, in Moyamba district, 88 out of 94 reporting orgunits have completed the PHUF1 form for January 2011. I can then select Moyamba as top orgunit, and will get the same statistics for the next level, in this case chiefdoms. When I finally select a chiefdom, and update my report, I get the individual facilties that have and have not reported.

It's a bit cumbersome, but if the data managers and district supervisors are also on DHIS2, it should be enough for top-level management to delegate the micro-management to them, and just send reminders like "Moyamba district is lacking PHUF1 for January from 6 facilities". In Sierra Leone we stressed that checking the completeness in DHIS2 was part of the district responsibilities.

Regards,
Johan

···

On 21.03.2012 08:14, Wilson,Randy wrote:

Hi all,

1.Data completeness report: There does not seem to be a report that
lists the orgunits that did not report for a given period for a given
dataset. We need this to issue reminders to data managers and their
district supervisors so that data entry is complete.

Another way of doing it (albeit with limitations) is through the data
browser. For instance,

Organisation Unit 2012W10 2012W11 2012W12
ce Central Province 289 289 221
co Copperbelt Province 0 0 0

This provides a count of the number of data elements which have been
submitted by province. From this picture, we can see that no data has
been submitted by Central Province for Weeks 10,11, and 12.
It looks like some data is missing from W12 from Central province,
because the number is lower. This is a quick and easy way to get a
feeling where there is data, and where there is not data, but a custom
report is probably a much better way to go.

Regards,
Jason

···

On Wed, Mar 21, 2012 at 9:55 AM, Johan Sæbø <johansa@ifi.uio.no> wrote:

On 21.03.2012 08:14, Wilson,Randy wrote:

Hi all,

1.Data completeness report: There does not seem to be a report that

lists the orgunits that did not report for a given period for a given
dataset. We need this to issue reminders to data managers and their
district supervisors so that data entry is complete.

Hi Randy, there's no in-built report for this, but it should be possible to
build a custom report as Jason suggests. However, you can find the orgunits
if you want, if you drill down in the Reporting Rate Summary page of
Reports.

Example from online demo (Sierra Leone orgunits):
I select top orgunit (Sierra Leone), and the dataset and period. I then get
the number of reporting units for each of the next level (district) that
should report (target) and those that actually have reported (actual).

For instance, in Moyamba district, 88 out of 94 reporting orgunits have
completed the PHUF1 form for January 2011. I can then select Moyamba as top
orgunit, and will get the same statistics for the next level, in this case
chiefdoms. When I finally select a chiefdom, and update my report, I get the
individual facilties that have and have not reported.

It's a bit cumbersome, but if the data managers and district supervisors are
also on DHIS2, it should be enough for top-level management to delegate the
micro-management to them, and just send reminders like "Moyamba district is
lacking PHUF1 for January from 6 facilities". In Sierra Leone we stressed
that checking the completeness in DHIS2 was part of the district
responsibilities.

Regards,
Johan

_______________________________________________
Mailing list: DHIS 2 Users in Launchpad
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : DHIS 2 Users in Launchpad
More help : ListHelp - Launchpad Help

That’s great.

Mydatamart is a tool that allows the users to have local copies of the server’s datamart with all the relevant local data stored on the user’s machine. This data can then be used in offline data analysis tools like the Exel PivotTable.

Ola

···

On Mar 21, 2012 12:06 AM, “Mark Spohr” mhspohr@gmail.com wrote:

We’ve got data!!

Thanks everyone for your help. I was confused and thought the Datamart utility was for the external program MyDatamart. I ran it and now my reports have actual data in them. This is a big improvement. I’ve set up scheduling to keep it up to date.

I also love the Data Visualizer… quite posh!

Thanks again,
Mark

On Tue, Mar 20, 2012 at 3:02 AM, Ola Hodne Titlestad olati@ifi.uio.no wrote:

HI Mark,

(copying dev list also since many people ask about this)

Here are datamart instructions for version 2.7:

Automatic datamart:

(2.6 and 2.7)

For servers we recommend you use the automatic scheduled datamart (runs at midnight every day).

This can be configured in Data Administration-Scheduling. (see user manual section 19.19)

It is correctly set up when it says “Scheduling is running” - which means that DHIS is scheduled to run a datamart export the next (and every) midnight. No need to click on Execute now (that starts the datamart process immediately, but this is sometimes needed as well)

Note that this scheduled datamart will export maximum data for 12 months back in time, (but will not delete older data). If you have an empty datamart and have data older than 12 months you should use the manual approach (see below) for those months that are older than 1 year.

How fast this scheduled datamart export process is will really depend on your CPU and Memory, but it will most likely take a while. In the tomcat log you can follow progress and CPU and memory use etc.

Manual datamart:

(this is or 2.7)

If you need to do manual datamart exports these are now possible from Reports->Datamart, but this is more simple now, it takes all data within the period you specify. This has been made much easier now.

(see section 13.2.1 in the user manual, it has been updated to this new datamart window).

(2.6)

A bit more complicated UI for datamart export accessed under Services->Datamart

Here you can select exactly what you want to export. Note that this export (2.6 and earlier) does not rebuild indexes on the datamart tables.

After an export, go to Maintenance->Data Administration->Maintenance: and tick Rebuild datamart indexes. Without that last step the reports will be a lot slower, especially on a multi-user environment.

Ola



Ola Hodne Titlestad (Mr)
HISP
Department of Informatics
University of Oslo

Mobile: +47 48069736
Home address: Vetlandsvn. 95B, 0685 Oslo, Norway. Googlemaps link

On 20 March 2012 09:49, Shinichi Suzuki shin461@gmail.com wrote:

Report data will generate from entered data by nightly batch job, usually.

You must go to data administration module, then “Scheduling”. It should be on automatically. But it might run out of memory as it uses a lot of resources.

Please try this.

         Suzuki

From: dhis2-users-bounces+shin461=gmail.com@lists.launchpad.net [mailto:dhis2-users-bounces+shin461=gmail.com@lists.launchpad.net] On Behalf Of Mark Spohr

Sent: Tuesday, March 20, 2012 8:00 AM
To: dhis2-users@lists.launchpad.net

Subject: [Dhis2-users] Reports - where’s the data?

Ok, I’m feeling quite stupid here.
We’ve been entering monthly data for 2011 with a form and we can view this using the “Dataset” report and it aggregates for province and for quarters and years.

Now, I want to put together some reports.
I try the Standard report… this requires a Table.
I put together a Table report and run it… nice table with data elements, org units and periods… but no data! (blank, nada).

I look at the web Pivot tables and run a few… nice pivot tables, rows, columns, periods… no data!.. blanks… not even a single number anywhere!

What am I missing here?

.Mark

Mark Spohr, MD
mhspohr@gmail.com
+1 530 554 2230


Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp


Mark Spohr, MD
mhspohr@gmail.com
+1 530 554 2230

Mind you I do sympathize with the confusion. I think it is the terminology of “exporting” the datamart which is used for generation of the server datamart which is too close to the terminology of exporting the piece of the datamart which is the mydatamart feed. The two are quite distinct but its easy to see how it might be confusing.

···

On 21 March 2012 10:38, Ola Hodne Titlestad olati@ifi.uio.no wrote:

That’s great.

Mydatamart is a tool that allows the users to have local copies of the server’s datamart with all the relevant local data stored on the user’s machine. This data can then be used in offline data analysis tools like the Exel PivotTable.

Ola

On Mar 21, 2012 12:06 AM, “Mark Spohr” mhspohr@gmail.com wrote:

We’ve got data!!

Thanks everyone for your help. I was confused and thought the Datamart utility was for the external program MyDatamart. I ran it and now my reports have actual data in them. This is a big improvement. I’ve set up scheduling to keep it up to date.

I also love the Data Visualizer… quite posh!

Thanks again,
Mark

On Tue, Mar 20, 2012 at 3:02 AM, Ola Hodne Titlestad olati@ifi.uio.no wrote:

HI Mark,

(copying dev list also since many people ask about this)

Here are datamart instructions for version 2.7:

Automatic datamart:

(2.6 and 2.7)

For servers we recommend you use the automatic scheduled datamart (runs at midnight every day).

This can be configured in Data Administration-Scheduling. (see user manual section 19.19)

It is correctly set up when it says “Scheduling is running” - which means that DHIS is scheduled to run a datamart export the next (and every) midnight. No need to click on Execute now (that starts the datamart process immediately, but this is sometimes needed as well)

Note that this scheduled datamart will export maximum data for 12 months back in time, (but will not delete older data). If you have an empty datamart and have data older than 12 months you should use the manual approach (see below) for those months that are older than 1 year.

How fast this scheduled datamart export process is will really depend on your CPU and Memory, but it will most likely take a while. In the tomcat log you can follow progress and CPU and memory use etc.

Manual datamart:

(this is or 2.7)

If you need to do manual datamart exports these are now possible from Reports->Datamart, but this is more simple now, it takes all data within the period you specify. This has been made much easier now.

(see section 13.2.1 in the user manual, it has been updated to this new datamart window).

(2.6)

A bit more complicated UI for datamart export accessed under Services->Datamart

Here you can select exactly what you want to export. Note that this export (2.6 and earlier) does not rebuild indexes on the datamart tables.

After an export, go to Maintenance->Data Administration->Maintenance: and tick Rebuild datamart indexes. Without that last step the reports will be a lot slower, especially on a multi-user environment.

Ola



Ola Hodne Titlestad (Mr)
HISP
Department of Informatics
University of Oslo

Mobile: +47 48069736
Home address: Vetlandsvn. 95B, 0685 Oslo, Norway. Googlemaps link

On 20 March 2012 09:49, Shinichi Suzuki shin461@gmail.com wrote:

Report data will generate from entered data by nightly batch job, usually.

You must go to data administration module, then “Scheduling”. It should be on automatically. But it might run out of memory as it uses a lot of resources.

Please try this.

         Suzuki

From: dhis2-users-bounces+shin461=gmail.com@lists.launchpad.net [mailto:dhis2-users-bounces+shin461=gmail.com@lists.launchpad.net] On Behalf Of Mark Spohr

Sent: Tuesday, March 20, 2012 8:00 AM
To: dhis2-users@lists.launchpad.net

Subject: [Dhis2-users] Reports - where’s the data?

Ok, I’m feeling quite stupid here.
We’ve been entering monthly data for 2011 with a form and we can view this using the “Dataset” report and it aggregates for province and for quarters and years.

Now, I want to put together some reports.
I try the Standard report… this requires a Table.
I put together a Table report and run it… nice table with data elements, org units and periods… but no data! (blank, nada).

I look at the web Pivot tables and run a few… nice pivot tables, rows, columns, periods… no data!.. blanks… not even a single number anywhere!

What am I missing here?

.Mark

Mark Spohr, MD
mhspohr@gmail.com
+1 530 554 2230


Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp


Mark Spohr, MD
mhspohr@gmail.com
+1 530 554 2230


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp