Program data into dataset

Hi everyone,

Since the aggregation queries are not there anymore is it possible to get program indicator value into an existing Aggregate Dataset through data entry screen design?

Or any other alternative workaround?

Regards

Pramil

Hi Pramil,

you can retrieve aggregated event data from the analytics api.

http://dhis2.github.io/dhis2-docs/master/en/developer/html/ch01s32.html

Excerpt from the docs:

To query for program data elements (of tracker domain type) you can get those by specifying the program for each data element using the . syntax:

/api/analytics.json?dimension=dx:eBAyeGv0exc.qrur9Dvnyt5;eBAyeGv0exc.GieVkTxp4HH &dimension=pe:LAST_12_MONTHS&filter=ou:ImspTQPwCqd
···

One work-around could be to write a bit of Javascript and include that in the custom form to retrieve and display event based aggregated data inside the data entry form, if that is what you are looking to do.

I have been pondering for a while on adding a feature where you could specify tracker-type data elements in custom data entry forms, and have the system automatically display aggregated event data, both in forms and in data set report. Let me know if that could work.

regards,

Lars

On Thu, May 5, 2016 at 11:03 AM, Pramil Liyanage pramilcl@gmail.com wrote:

Hi everyone,

Since the aggregation queries are not there anymore is it possible to get program indicator value into an existing Aggregate Dataset through data entry screen design?

Or any other alternative workaround?

Regards

Pramil


Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp

Lars Helge Øverland

Lead developer, DHIS 2

University of Oslo

Skype: larshelgeoverland

lars@dhis2.org

http://www.dhis2.org

Dear Lars,

Making aggregated event data available on the fly in data set report would certainly be a useful tool especially when maintaining monitoring and evaluation forms for tracker activities.

Thank you

···

On Fri, May 6, 2016 at 6:35 PM, Lars Helge Øverland lars@dhis2.org wrote:

Hi Pramil,

you can retrieve aggregated event data from the analytics api.

http://dhis2.github.io/dhis2-docs/master/en/developer/html/ch01s32.html

Excerpt from the docs:

To query for program data elements (of tracker domain type) you can get those by specifying the program for each data element using the . syntax:

/api/analytics.json?dimension=dx:eBAyeGv0exc.qrur9Dvnyt5;eBAyeGv0exc.GieVkTxp4HH &dimension=pe:LAST_12_MONTHS&filter=ou:ImspTQPwCqd

Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

One work-around could be to write a bit of Javascript and include that in the custom form to retrieve and display event based aggregated data inside the data entry form, if that is what you are looking to do.

I have been pondering for a while on adding a feature where you could specify tracker-type data elements in custom data entry forms, and have the system automatically display aggregated event data, both in forms and in data set report. Let me know if that could work.

regards,

Lars

On Thu, May 5, 2016 at 11:03 AM, Pramil Liyanage pramilcl@gmail.com wrote:

Hi everyone,

Since the aggregation queries are not there anymore is it possible to get program indicator value into an existing Aggregate Dataset through data entry screen design?

Or any other alternative workaround?

Regards

Pramil


Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp

Lars Helge Øverland

Lead developer, DHIS 2

University of Oslo

Skype: larshelgeoverland

lars@dhis2.org

http://www.dhis2.org

Regards,
Pamod Amarakoon

Confidentiality Notice: the information contained in this email and any attachments may be legally privileged and confidential. If you are not an intended recipient, you are hereby notified that any dissemination, distribution, or copying of this e-mail is strictly prohibited. If you have received this e-mail in error, please notify the sender and permanently delete the e-mail and any attachments immediately. You should not retain, copy or use this e-mail or any attachments for any purpose, nor disclose all or any part of the contents to any other person.

Thanks Lars. Let me summarize my issue. We have been using aggregated reporting for TB for a few years and recently introduced individual records and aggregated using query builder to the same aggregated dataset. This was customized to the exact national paper report and report was generated from dataset report for printing.

Now since agg. queries are not there we will not be able to get data from tracker into the same data element. If we shift to a new report using program indicators we will not be able to use the previous aggregate data in future analysis and comparison.

To make the matter worse, there was a lot of disaggregation of data elements using categories, where we wrote queries for each one.

We are stuck at a dead end at 2.22! Please suggest the best solution.

Your suggestion to have program indicators in the custom dataset is a good idea for future. But my issue is get them into existing elements if I upgrade to 2.23, and not a few, about 700 in all reports.

Regards

Pramil

···

On Fri, May 6, 2016 at 6:55 PM, Pamod Amarakoon pamodm@gmail.com wrote:

Dear Lars,

Making aggregated event data available on the fly in data set report would certainly be a useful tool especially when maintaining monitoring and evaluation forms for tracker activities.

Thank you

On Fri, May 6, 2016 at 6:35 PM, Lars Helge Øverland lars@dhis2.org wrote:

Hi Pramil,

you can retrieve aggregated event data from the analytics api.

http://dhis2.github.io/dhis2-docs/master/en/developer/html/ch01s32.html

Excerpt from the docs:

To query for program data elements (of tracker domain type) you can get those by specifying the program for each data element using the . syntax:

/api/analytics.json?dimension=dx:eBAyeGv0exc.qrur9Dvnyt5;eBAyeGv0exc.GieVkTxp4HH &dimension=pe:LAST_12_MONTHS&filter=ou:ImspTQPwCqd

Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Regards,
Pamod Amarakoon

Confidentiality Notice: the information contained in this email and any attachments may be legally privileged and confidential. If you are not an intended recipient, you are hereby notified that any dissemination, distribution, or copying of this e-mail is strictly prohibited. If you have received this e-mail in error, please notify the sender and permanently delete the e-mail and any attachments immediately. You should not retain, copy or use this e-mail or any attachments for any purpose, nor disclose all or any part of the contents to any other person.

One work-around could be to write a bit of Javascript and include that in the custom form to retrieve and display event based aggregated data inside the data entry form, if that is what you are looking to do.

I have been pondering for a while on adding a feature where you could specify tracker-type data elements in custom data entry forms, and have the system automatically display aggregated event data, both in forms and in data set report. Let me know if that could work.

regards,

Lars

On Thu, May 5, 2016 at 11:03 AM, Pramil Liyanage pramilcl@gmail.com wrote:

Hi everyone,

Since the aggregation queries are not there anymore is it possible to get program indicator value into an existing Aggregate Dataset through data entry screen design?

Or any other alternative workaround?

Regards

Pramil


Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp

Lars Helge Øverland

Lead developer, DHIS 2

University of Oslo

Skype: larshelgeoverland

lars@dhis2.org

http://www.dhis2.org

Any help on this?

Regards

Pramil

···

On Sat, May 7, 2016 at 8:53 PM, Pramil Liyanage pramilcl@gmail.com wrote:

Thanks Lars. Let me summarize my issue. We have been using aggregated reporting for TB for a few years and recently introduced individual records and aggregated using query builder to the same aggregated dataset. This was customized to the exact national paper report and report was generated from dataset report for printing.

Now since agg. queries are not there we will not be able to get data from tracker into the same data element. If we shift to a new report using program indicators we will not be able to use the previous aggregate data in future analysis and comparison.

To make the matter worse, there was a lot of disaggregation of data elements using categories, where we wrote queries for each one.

We are stuck at a dead end at 2.22! Please suggest the best solution.

Your suggestion to have program indicators in the custom dataset is a good idea for future. But my issue is get them into existing elements if I upgrade to 2.23, and not a few, about 700 in all reports.

Regards

Pramil

On Fri, May 6, 2016 at 6:55 PM, Pamod Amarakoon pamodm@gmail.com wrote:

Dear Lars,

Making aggregated event data available on the fly in data set report would certainly be a useful tool especially when maintaining monitoring and evaluation forms for tracker activities.

Thank you

On Fri, May 6, 2016 at 6:35 PM, Lars Helge Øverland lars@dhis2.org wrote:

Hi Pramil,

you can retrieve aggregated event data from the analytics api.

http://dhis2.github.io/dhis2-docs/master/en/developer/html/ch01s32.html

Excerpt from the docs:

To query for program data elements (of tracker domain type) you can get those by specifying the program for each data element using the . syntax:

/api/analytics.json?dimension=dx:eBAyeGv0exc.qrur9Dvnyt5;eBAyeGv0exc.GieVkTxp4HH &dimension=pe:LAST_12_MONTHS&filter=ou:ImspTQPwCqd

Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Regards,
Pamod Amarakoon

Confidentiality Notice: the information contained in this email and any attachments may be legally privileged and confidential. If you are not an intended recipient, you are hereby notified that any dissemination, distribution, or copying of this e-mail is strictly prohibited. If you have received this e-mail in error, please notify the sender and permanently delete the e-mail and any attachments immediately. You should not retain, copy or use this e-mail or any attachments for any purpose, nor disclose all or any part of the contents to any other person.

One work-around could be to write a bit of Javascript and include that in the custom form to retrieve and display event based aggregated data inside the data entry form, if that is what you are looking to do.

I have been pondering for a while on adding a feature where you could specify tracker-type data elements in custom data entry forms, and have the system automatically display aggregated event data, both in forms and in data set report. Let me know if that could work.

regards,

Lars

On Thu, May 5, 2016 at 11:03 AM, Pramil Liyanage pramilcl@gmail.com wrote:

Hi everyone,

Since the aggregation queries are not there anymore is it possible to get program indicator value into an existing Aggregate Dataset through data entry screen design?

Or any other alternative workaround?

Regards

Pramil


Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp

Lars Helge Øverland

Lead developer, DHIS 2

University of Oslo

Skype: larshelgeoverland

lars@dhis2.org

http://www.dhis2.org

Hi Pramil & Lars,

actually I think that Pramil is adressing a very valid problem. In my current
understanding of DHIS2 I wouldn't be able to propose any better solution than
using Lars' analysis api and an external tool (php, r, kettle etc) to read the
results and fill them into the datasets.

However, the use-case he describes is the very typical data warehousing
scenario, where you have different layers of granularity and maturity of data
(e.g. staging area, operational data stores, data marts), especially in an
infrastructure where you are mainly extracting data from source systems. This
would require DHIS2 to be able to push data internally (from tracker to
datasets, but also within tracker or from dataset to dataset), while at the same
time being able to transform these data (e.g. as it was possible with the
aggregation queries).

This has multiple semantic advantages as you can model your data from raw-data
to architected data marts. But also technical aspects as you don't have to
rebuild your datasets from the source systems, whenever you change their
definition, but can rebuild them from the granular level instead. Leave alone
the performance issues I'd normally expect when having to include huge numbers
of granular data from tracker into DHIS2 analytics. (btw. do you have any
reference data on the that?) Of course this would call for nightly builds, but
that's normal data warehousing business.

I am aware of the discussion last year (e.g.
https://lists.launchpad.net/dhis2-users/msg09609.html) but would like to
encourage a higher priority of the replacement, if there is still the intention
of positioning DHIS2 as a tool for data warehousing.

I also remember slightly that Knut tried to trigger off a discussion on this
(maybe around 2015-12?), but can't find the thread on that - are there any DEV
internal news on the status of that discussion and the future plans of
aggregation within DHIS2?

Best regards,

Uwe

···

---

Pramil Liyanage <pramilcl@gmail.com> hat am 9. Mai 2016 um 09:00 geschrieben:

Any help on this?

Regards
Pramil

On Sat, May 7, 2016 at 8:53 PM, Pramil Liyanage <pramilcl@gmail.com> wrote:

> Thanks Lars. Let me summarize my issue. We have been using aggregated
> reporting for TB for a few years and recently introduced individual records
> and aggregated using query builder to the same aggregated dataset. This was
> customized to the exact national paper report and report was generated from
> dataset report for printing.
>
> Now since agg. queries are not there we will not be able to get data from
> tracker into the same data element. If we shift to a new report using
> program indicators we will not be able to use the previous aggregate data
> in future analysis and comparison.
>
> To make the matter worse, there was a lot of disaggregation of data
> elements using categories, where we wrote queries for each one.
>
> We are stuck at a dead end at 2.22! Please suggest the best solution.
>
> Your suggestion to have program indicators in the custom dataset is a good
> idea for future. But my issue is get them into existing elements if I
> upgrade to 2.23, and not a few, about 700 in all reports.
>
> Regards
> Pramil
>
>
>
>
>
> On Fri, May 6, 2016 at 6:55 PM, Pamod Amarakoon <pamodm@gmail.com> wrote:
>
>> Dear Lars,
>>
>> Making aggregated event data available on the fly in data set report
>> would certainly be a useful tool especially when maintaining monitoring and
>> evaluation forms for tracker activities.
>>
>> Thank you
>>
>> On Fri, May 6, 2016 at 6:35 PM, Lars Helge Øverland <lars@dhis2.org> > >> wrote:
>>
>>> Hi Pramil,
>>>
>>> you can retrieve aggregated event data from the analytics api.
>>>
>>> http://dhis2.github.io/dhis2-docs/master/en/developer/html/ch01s32.html
>>>
>>>
>>> Excerpt from the docs:
>>>
>>> To query for program data elements (of tracker domain type) you can get
>>> those by specifying the program for each data element using the
>>> <program-id>.<dataelement-id> syntax:
>>>
>>> /api/analytics.json?dimension=dx:eBAyeGv0exc.qrur9Dvnyt5;eBAyeGv0exc.GieVkTxp4HH
>>> &dimension=pe:LAST_12_MONTHS&filter=ou:ImspTQPwCqd
>>>
>>> One work-around could be to write a bit of Javascript and include that
>>> in the custom form to retrieve and display event based aggregated data
>>> inside the data entry form, if that is what you are looking to do.
>>>
>>>
>>> I have been pondering for a while on adding a feature where you could
>>> specify tracker-type data elements in custom data entry forms, and have
>>> the
>>> system automatically display aggregated event data, both in forms and in
>>> data set report. Let me know if that could work.
>>>
>>>
>>> regards,
>>>
>>> Lars
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Thu, May 5, 2016 at 11:03 AM, Pramil Liyanage <pramilcl@gmail.com> > >>> wrote:
>>>
>>>> Hi everyone,
>>>>
>>>> Since the aggregation queries are not there anymore is it possible to
>>>> get program indicator value into an existing Aggregate Dataset through
>>>> data
>>>> entry screen design?
>>>>
>>>> Or any other alternative workaround?
>>>>
>>>> Regards
>>>> Pramil
>>>>
>>>> _______________________________________________
>>>> Mailing list: https://launchpad.net/~dhis2-users
>>>> Post to : dhis2-users@lists.launchpad.net
>>>> Unsubscribe : https://launchpad.net/~dhis2-users
>>>> More help : https://help.launchpad.net/ListHelp
>>>>
>>>>
>>>
>>>
>>> --
>>> Lars Helge Øverland
>>> Lead developer, DHIS 2
>>> University of Oslo
>>> Skype: larshelgeoverland
>>> lars@dhis2.org
>>> http://www.dhis2.org <https://www.dhis2.org/>
>>>
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~dhis2-devs
>>> Post to : dhis2-devs@lists.launchpad.net
>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>> More help : https://help.launchpad.net/ListHelp
>>>
>>>
>>
>>
>> --
>> Regards,
>> Pamod Amarakoon
>>
>> Confidentiality Notice: the information contained in this email and any
>> attachments may be legally privileged and confidential. If you are not an
>> intended recipient, you are hereby notified that any dissemination,
>> distribution, or copying of this e-mail is strictly prohibited. If you have
>> received this e-mail in error, please notify the sender and permanently
>> delete the e-mail and any attachments immediately. You should not retain,
>> copy or use this e-mail or any attachments for any purpose, nor disclose
>> all or any part of the contents to any other person.
>>
>>
>>
>>
>
_______________________________________________
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp

Hi Uwe and Lars,

Thanks for highlighting the importance. As Uwe said, if DHIS2 to continue as a data warehouse, aggregating from tracker will be important. I’m sure there are many who is having the same issue. And in future also there will be many who will face this when moving into patient records as further developments.

Hope the developers will provide a better solution soon.

Regards

Pramil

···

On Mon, May 9, 2016 at 1:11 PM, Uwe Wahser uwe@wahser.de wrote:

Hi Pramil & Lars,

actually I think that Pramil is adressing a very valid problem. In my current

understanding of DHIS2 I wouldn’t be able to propose any better solution than

using Lars’ analysis api and an external tool (php, r, kettle etc) to read the

results and fill them into the datasets.

However, the use-case he describes is the very typical data warehousing

scenario, where you have different layers of granularity and maturity of data

(e.g. staging area, operational data stores, data marts), especially in an

infrastructure where you are mainly extracting data from source systems. This

would require DHIS2 to be able to push data internally (from tracker to

datasets, but also within tracker or from dataset to dataset), while at the same

time being able to transform these data (e.g. as it was possible with the

aggregation queries).

This has multiple semantic advantages as you can model your data from raw-data

to architected data marts. But also technical aspects as you don’t have to

rebuild your datasets from the source systems, whenever you change their

definition, but can rebuild them from the granular level instead. Leave alone

the performance issues I’d normally expect when having to include huge numbers

of granular data from tracker into DHIS2 analytics. (btw. do you have any

reference data on the that?) Of course this would call for nightly builds, but

that’s normal data warehousing business.

I am aware of the discussion last year (e.g.

https://lists.launchpad.net/dhis2-users/msg09609.html) but would like to

encourage a higher priority of the replacement, if there is still the intention

of positioning DHIS2 as a tool for data warehousing.

I also remember slightly that Knut tried to trigger off a discussion on this

(maybe around 2015-12?), but can’t find the thread on that - are there any DEV

internal news on the status of that discussion and the future plans of

aggregation within DHIS2?

Best regards,

Uwe


Pramil Liyanage pramilcl@gmail.com hat am 9. Mai 2016 um 09:00 geschrieben:

Any help on this?

Regards

Pramil

On Sat, May 7, 2016 at 8:53 PM, Pramil Liyanage pramilcl@gmail.com wrote:

Thanks Lars. Let me summarize my issue. We have been using aggregated

reporting for TB for a few years and recently introduced individual records

and aggregated using query builder to the same aggregated dataset. This was

customized to the exact national paper report and report was generated from

dataset report for printing.

Now since agg. queries are not there we will not be able to get data from

tracker into the same data element. If we shift to a new report using

program indicators we will not be able to use the previous aggregate data

in future analysis and comparison.

To make the matter worse, there was a lot of disaggregation of data

elements using categories, where we wrote queries for each one.

We are stuck at a dead end at 2.22! Please suggest the best solution.

Your suggestion to have program indicators in the custom dataset is a good

idea for future. But my issue is get them into existing elements if I

upgrade to 2.23, and not a few, about 700 in all reports.

Regards

Pramil

On Fri, May 6, 2016 at 6:55 PM, Pamod Amarakoon pamodm@gmail.com wrote:

Dear Lars,

Making aggregated event data available on the fly in data set report

would certainly be a useful tool especially when maintaining monitoring and

evaluation forms for tracker activities.

Thank you

On Fri, May 6, 2016 at 6:35 PM, Lars Helge Øverland lars@dhis2.org

wrote:

Hi Pramil,

you can retrieve aggregated event data from the analytics api.

http://dhis2.github.io/dhis2-docs/master/en/developer/html/ch01s32.html

Excerpt from the docs:

To query for program data elements (of tracker domain type) you can get

those by specifying the program for each data element using the

. syntax:

/api/analytics.json?dimension=dx:eBAyeGv0exc.qrur9Dvnyt5;eBAyeGv0exc.GieVkTxp4HH

&dimension=pe:LAST_12_MONTHS&filter=ou:ImspTQPwCqd

One work-around could be to write a bit of Javascript and include that

in the custom form to retrieve and display event based aggregated data

inside the data entry form, if that is what you are looking to do.

I have been pondering for a while on adding a feature where you could

specify tracker-type data elements in custom data entry forms, and have

the

system automatically display aggregated event data, both in forms and in

data set report. Let me know if that could work.

regards,

Lars

On Thu, May 5, 2016 at 11:03 AM, Pramil Liyanage pramilcl@gmail.com

wrote:

Hi everyone,

Since the aggregation queries are not there anymore is it possible to

get program indicator value into an existing Aggregate Dataset through

data

entry screen design?

Or any other alternative workaround?

Regards

Pramil


Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp

Lars Helge Øverland

Lead developer, DHIS 2

University of Oslo

Skype: larshelgeoverland

lars@dhis2.org

http://www.dhis2.org <https://www.dhis2.org/>


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Regards,

Pamod Amarakoon

Confidentiality Notice: the information contained in this email and any

attachments may be legally privileged and confidential. If you are not an

intended recipient, you are hereby notified that any dissemination,

distribution, or copying of this e-mail is strictly prohibited. If you have

received this e-mail in error, please notify the sender and permanently

delete the e-mail and any attachments immediately. You should not retain,

copy or use this e-mail or any attachments for any purpose, nor disclose

all or any part of the contents to any other person.


Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp

I kown that is a 5 year old thread, but google showed it to me in the first choices so it seems still interest people

I found an app that seems relevant, here the dedicated post Program Dataset Connector: PDAC

1 Like