Running integrity checks on datasets with unlinked data element?

Dear jason,
i have custom forms!

------------------------------On Wed, Jan 11, 2012 11:54 AM PST Jason Pickering wrote:>Seems like a good idea. Do you have section forms or custom forms?>>>On Wed, Jan 11, 2012 at 9:18 PM, Muhire Andrew <muhireandrew@yahoo.com> wrote:>>>> Hi to all,>> i would like to run integrity check to see the data element that are in dataset that are not linked to the form. Is this possible? Not data element without dataset ( data element in dataset that is not linked to the form). Because this affects data completeness report(based on data values) where by the target column counts all linked and nonlinked data elements which can confuse users. Per now i just use a manual checking.>>>>>>>>>>>>>>>------------------------------On Wed, Jan 11, 2012 2:06 AM PST Ola Hodne Titlestad wrote:>I would suggest to capture service availability and stock-outs as separate>data elements or even orgunit groups for services, and not as some>interpretation of a 0. Then there is no
need to capture zeros.>Once the data is aggregated up one level all this zero/blank interpretation>is impossible anyway. The complete button also gives an indication of>wheter data is missing of not.>Ola>On Jan 11, 2012 9:45 AM, "Knut Staring" <knutst@gmail.com> wrote:> Here in India, the activities of Subcenters differ quite a bit in the> particular services they offer. Therefore, since all use a standard format,> there will almost always be a number of fields that are not applicable.> Because of this, they maintain that 0 is signficant, because it means "Have> the service, but no cases this month", whereas a blank field means "Don't> have the service" in other words N/A.>>>> Sometimes it can even mean something in-between, like "We normally> distribute Iron tablets, but sometimes we don't have them (blank), and> sometimes we have them but there is no demand (0)". In my view the latter> semantics is quite complicated, and hard to get consistent, but
I'm being> told the health workers handle it well. Thus, a typical report can consist> of 50 positive integers, 7 blanks and 20 zeros.> Since data entry on the mobile is quite cumbersome and zeros also tend to> fill up the database, I would have preferred zeros to be non-significant> and just go with blanks. But then we would lose the above distinction.> Knut>> --> Knut Staring> Dept. of Informatics, University of Oslo> +4791880522> http://dhis2.org>> _______________________________________________> Mailing list: DHIS 2 developers in Launchpad> Post to : dhis2-devs@lists.launchpad.net> Unsubscribe : DHIS 2 developers in Launchpad> More help > :>> ListHelp - Launchpad Help>>>>> _______________________________________________>> Mailing list: DHIS 2 Users in Launchpad>> Post to : dhis2-users@lists.launchpad.net>> Unsubscribe : DHIS 2 Users in Launchpad>> More help : ListHelp - Launchpad Help

This may be yet another good reason to try and move away from custom forms. I think most of your forms could work well as custom forms.

Knut

···

On Thursday, January 12, 2012, Muhire Andrew wrote:

Dear jason,

i have custom forms!

------------------------------On Wed, Jan 11, 2012 11:54 AM PST Jason Pickering wrote:>Seems like a good idea. Do you have section forms or custom forms?>>>On Wed, Jan 11, 2012 at 9:18 PM, Muhire Andrew muhireandrew@yahoo.com wrote:>>>> Hi to all,>> i would like to run integrity check to see the data element that are in dataset that are not linked to the form. Is this possible? Not data element without dataset ( data element in dataset that is not linked to the form). Because this affects data completeness report(based on data values) where by the target column counts all linked and nonlinked data elements which can confuse users. Per now i just use a manual checking.>>>>>>>>>>>>>>>------------------------------On Wed, Jan 11, 2012 2:06 AM PST Ola Hodne Titlestad wrote:>I would suggest to capture service availability and stock-outs as separate>data elements or even orgunit groups for services, and not as some>interpretation of a 0. Then there is no

need to capture zeros.>Once the data is aggregated up one level all this zero/blank interpretation>is impossible anyway. The complete button also gives an indication of>wheter data is missing of not.>Ola>On Jan 11, 2012 9:45 AM, “Knut Staring” knutst@gmail.com wrote:> Here in India, the activities of Subcenters differ quite a bit in the> particular services they offer. Therefore, since all use a standard format,> there will almost always be a number of fields that are not applicable.> Because of this, they maintain that 0 is signficant, because it means “Have> the service, but no cases this month”, whereas a blank field means “Don’t> have the service” in other words N/A.>>>> Sometimes it can even mean something in-between, like “We normally> distribute Iron tablets, but sometimes we don’t have them (blank), and> sometimes we have them but there is no demand (0)”. In my view the latter> semantics is quite complicated, and hard to get consistent, but

I’m being> told the health workers handle it well. Thus, a typical report can consist> of 50 positive integers, 7 blanks and 20 zeros.> Since data entry on the mobile is quite cumbersome and zeros also tend to> fill up the database, I would have preferred zeros to be non-significant> and just go with blanks. But then we would lose the above distinction.> Knut>> --> Knut Staring> Dept. of Informatics, University of Oslo> +4791880522> http://dhis2.org>> _______________________________________________> Mailing list: https://launchpad.net/~dhis2-devs> Post to : dhis2-devs@lists.launchpad.net> Unsubscribe : https://launchpad.net/~dhis2-devs> More help > :>> https://help.launchpad.net/ListHelp>>>>> _______________________________________________>> Mailing list: https://launchpad.net/~dhis2-users>> Post to : dhis2-users@lists.launchpad.net>> Unsubscribe : https://launchpad.net/~dhis2-users>> More help : https://help.launchpad.net/ListHelp


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp


Knut Staring

Dept. of Informatics, University of Oslo

+4791880522

http://dhis2.org

I have an idea how this could theoretically be done, but it would not
be so simple SQL as is normally used for the data integrity checks.
The custom forms are stored as HTML, and we would need to somehow
strip out all of the dataelements/catcombooptions and compare these
against what is "supposed" to be there. The problem is that custom
forms I think do not really have a requirement to have all the data
elements in a data set. For instance, I have left out some
DE/catcombooptions which are never used for data entry (similar to the
grey fields of the section forms), so that data is never entered into
them.

I would tend to agree with Knut. If you use the section forms, you can
be pretty sure that all of your data elements in the dataset are going
to show up.

···

On Thu, Jan 12, 2012 at 5:16 AM, Knut Staring <knutst@gmail.com> wrote:

This may be yet another good reason to try and move away from custom forms.
I think most of your forms could work well as custom forms.

Knut

On Thursday, January 12, 2012, Muhire Andrew wrote:

Dear jason,
i have custom forms!

------------------------------On Wed, Jan 11, 2012 11:54 AM PST Jason
Pickering wrote:>Seems like a good idea. Do you have section forms or custom
forms?>>>On Wed, Jan 11, 2012 at 9:18 PM, Muhire Andrew
<muhireandrew@yahoo.com> wrote:>>>> Hi to all,>> i would like to run
integrity check to see the data element that are in dataset that are not
linked to the form. Is this possible? Not data element without dataset (
data element in dataset that is not linked to the form). Because this
affects data completeness report(based on data values) where by the target
column counts all linked and nonlinked data elements which can confuse
users. Per now i just use a manual
checking.>>>>>>>>>>>>>>>------------------------------On Wed, Jan 11, 2012
2:06 AM PST Ola Hodne Titlestad wrote:>I would suggest to capture service
availability and stock-outs as separate>data elements or even orgunit groups
for services, and not as some>interpretation of a 0. Then there is no
need to capture zeros.>Once the data is aggregated up one level all this
zero/blank interpretation>is impossible anyway. The complete button also
gives an indication of>wheter data is missing of not.>Ola>On Jan 11, 2012
9:45 AM, "Knut Staring" <knutst@gmail.com> wrote:> Here in India, the
activities of Subcenters differ quite a bit in the> particular services
they offer. Therefore, since all use a standard format,> there will almost
always be a number of fields that are not applicable.> Because of this, they
maintain that 0 is signficant, because it means "Have> the service, but no
cases this month", whereas a blank field means "Don't> have the service" in
other words N/A.>>>> Sometimes it can even mean something in-between, like
"We normally> distribute Iron tablets, but sometimes we don't have them
(blank), and> sometimes we have them but there is no demand (0)". In my view
the latter> semantics is quite complicated, and hard to get consistent, but
I'm being> told the health workers handle it well. Thus, a typical report
can consist> of 50 positive integers, 7 blanks and 20 zeros.> Since data
entry on the mobile is quite cumbersome and zeros also tend to> fill up the
database, I would have preferred zeros to be non-significant> and just go
with blanks. But then we would lose the above distinction.> Knut>> --> Knut
> Dept. of Informatics, University of Oslo> +4791880522>
http://dhis2.org>> _______________________________________________> Mailing
list: DHIS 2 developers in Launchpad> Post to :
dhis2-devs@lists.launchpad.net> Unsubscribe :
DHIS 2 developers in Launchpad> More help > :>>
ListHelp - Launchpad Help>>>>>
_______________________________________________>> Mailing list:
DHIS 2 Users in Launchpad>> Post to :
dhis2-users@lists.launchpad.net>> Unsubscribe :
DHIS 2 Users in Launchpad>> More help :
ListHelp - Launchpad Help

_______________________________________________
Mailing list: DHIS 2 developers in Launchpad
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : DHIS 2 developers in Launchpad
More help : ListHelp - Launchpad Help

--
Knut Staring
Dept. of Informatics, University of Oslo
+4791880522
http://dhis2.org

_______________________________________________
Mailing list: DHIS 2 Users in Launchpad
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : DHIS 2 Users in Launchpad
More help : ListHelp - Launchpad Help

You are right dear Knut.

···

Muhire Andrew

HMIS/Ministry of Health

First say to yourself what you would be; and then do what you have to do..


From: Knut Staring knutst@gmail.com
To: Muhire Andrew muhireandrew@yahoo.com
Cc:
dhis2-users@lists.launchpad.netdhis2-users@lists.launchpad.net
Sent: Thursday, January 12, 2012 5:16 AM
Subject: Re: [Dhis2-users] Running integrity checks on datasets with unlinked data element?

This may be yet another good reason to try and move away from custom forms. I think most of your forms could work well as custom forms.

Knut

On Thursday, January 12, 2012, Muhire Andrew wrote:

Dear jason,

i have custom forms!

------------------------------On Wed, Jan 11, 2012 11:54 AM PST Jason Pickering wrote:>Seems like a good idea. Do you have section forms or custom forms?>>>On Wed, Jan 11, 2012 at 9:18 PM, Muhire Andrew <muhireandrew@yahoo.com > wrote:>>>> Hi to all,>> i would like to run integrity check to see the data element that are in dataset that are not linked to the form. Is this possible? Not data element without dataset ( data element in dataset that is not linked to the form). Because this affects data completeness report(based on data values) where by the target column counts all linked and nonlinked data elements which can confuse users. Per now i just use a manual checking.>>>>>>>>>>>>>>>------------------------------On Wed, Jan 11, 2012 2:06 AM PST Ola Hodne Titlestad wrote:>I would suggest to capture service availability and stock-outs as separate>data elements or even orgunit groups for services, and not as some>interpretation of a 0. Then there is no

need to capture zeros.>Once the data is aggregated up one level all this zero/blank interpretation>is impossible anyway. The complete button also gives an indication of>wheter data is missing of not.>Ola>On Jan 11, 2012 9:45 AM, “Knut Staring” <knutst@gmail.com > wrote:> Here in India, the activities of Subcenters differ quite a bit in the> particular services they offer. Therefore, since all use a standard format,> there will almost always be a number of fields that are not applicable.> Because of this, they maintain that 0 is signficant, because it means “Have> the service, but no cases this month”, whereas a blank field means “Don’t> have the service” in other words N/A.>>>> Sometimes it can even mean something in-between, like “We normally> distribute Iron tablets, but sometimes we don’t have them (blank), and> sometimes we have them but there is no demand (0)”. In my view the latter> semantics is quite complicated, and hard to get consistent, but

I’m being> told the health workers handle it well. Thus, a typical report can consist> of 50 positive integers, 7 blanks and 20 zeros.> Since data entry on the mobile is quite cumbersome and zeros also tend to> fill up the database, I would have preferred zeros to be non-significant> and just go with blanks. But then we would lose the above distinction.> Knut>> → Knut Staring> Dept. of Informatics, University of Oslo> +4791880522> http://dhis2.org>> _______________________________________________> Mailing list: https://launchpad.net/~dhis2-devs> Post to : dhis2-devs@lists.launchpad.net> Unsubscribe : https://launchpad.net/~dhis2-devs > More help > :>> https://help.launchpad.net/ListHelp>>>>> _______________________________________________>> Mailing list: https://launchpad.net/~dhis2-users>> Post to : dhis2-users@lists.launchpad.net>> Unsubscribe : https://launchpad.net/~dhis2-users>> More help : https://help.launchpad.net/ListHelp


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp


Knut Staring

Dept. of Informatics, University of Oslo

+4791880522

Thanks Jason! so section forms are recommended to be on a safe side.

···

Muhire Andrew

HMIS/Ministry of Health

First say to yourself what you would be; and then do what you have to do..


From: Jason Pickering jason.p.pickering@gmail.com
To: Knut Staring knutst@gmail.com
Cc: Muhire Andrew muhireandrew@yahoo.com; “dhis2-users@lists.launchpad.netdhis2-users@lists.launchpad.net
Sent: Thursday, January 12, 2012 6:46 AM
Subject: Re: [Dhis2-users] Running integrity checks on datasets with unlinked data element?

I have an idea how this could theoretically be done, but it would not
be so simple SQL as is normally used for the data integrity checks.
The custom forms are stored as HTML, and we would need to somehow
strip out all of the dataelements/catcombooptions and compare these
against what is “supposed” to be there. The problem is that custom
forms I think do not really have a requirement to have all the data
elements in a data set. For instance, I have left out some
DE/catcombooptions which are never used for data entry (similar to the
grey fields of the section forms), so that data is never entered into
them.

I would tend to agree with Knut. If you use the section forms, you can
be pretty sure that all of your data elements in the dataset are going
to show up.

On Thu, Jan 12, 2012 at 5:16 AM, Knut Staring <knutst@gmail.com > wrote:

This may be yet another good reason to try and move away from custom forms.
I think most of your forms could work well as custom forms.

Knut

On Thursday, January 12, 2012, Muhire Andrew wrote:

Dear jason,
i have custom forms!

------------------------------On Wed, Jan 11, 2012 11:54 AM PST Jason
Pickering wrote:>Seems like a good idea. Do you have section forms or custom
forms?>>>On Wed, Jan 11, 2012 at 9:18 PM, Muhire Andrew
muhireandrew@yahoo.com wrote:>>>> Hi to all,>> i would like to run
integrity check to see the data element that are in dataset that are not
linked to the form. Is this possible? Not data element without dataset (
data element in dataset that is not linked to the form). Because this
affects data completeness report(based on data values) where by the target
column counts all linked and nonlinked data elements which can confuse
users. Per now i just use a manual
checking.>>>>>>>>>>>>>>>------------------------------On Wed, Jan 11, 2012
2:06 AM PST Ola Hodne Titlestad wrote:>I would suggest to capture service
availability and stock-outs as separate>data elements or even orgunit groups
for services, and not as some>interpretation of a 0. Then there is no
need to capture zeros.>Once the data is aggregated up one level all this
zero/blank interpretation>is impossible anyway. The complete button also
gives an indication of>wheter data is missing of not.>Ola>On Jan 11, 2012
9:45 AM, “Knut Staring” knutst@gmail.com wrote:> Here in India, the
activities of Subcenters differ quite a bit in the> particular services
they offer. Therefore, since all use a standard format,> there will almost
always be a number of fields that are not applicable.> Because of this, they
maintain that 0 is signficant, because it means “Have> the service, but no
cases this month”, whereas a blank field means “Don’t> have the service” in
other words N/A.>>>> Sometimes it can even mean something in-between, like
“We normally> distribute Iron tablets, but sometimes we don’t have them
(blank), and> sometimes we have them but there is no demand (0)”. In my view
the latter> semantics is quite complicated, and hard to get consistent, but
I’m being> told the health workers handle it well. Thus, a typical report
can consist> of 50 positive integers, 7 blanks and 20 zeros.> Since data
entry on the mobile is quite cumbersome and zeros also tend to> fill up the
database, I would have preferred zeros to be non-significant> and just go
with blanks. But then we would lose the above distinction.> Knut>> → Knut

Dept. of Informatics, University of Oslo> +4791880522>
http://dhis2.org>> _______________________________________________> Mailing
list: https://launchpad.net/~dhis2-devs> Post to :
dhis2-devs@lists.launchpad.net> Unsubscribe :
https://launchpad.net/~dhis2-devs> More help > :>>
https://help.launchpad.net/ListHelp>>>>>
_______________________________________________>> Mailing list:
https://launchpad.net/~dhis2-users>> Post to :
dhis2-users@lists.launchpad.net>> Unsubscribe :
https://launchpad.net/~dhis2-users>> More help :
https://help.launchpad.net/ListHelp


Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help : https://help.launchpad.net/ListHelp


Knut Staring
Dept. of Informatics, University of Oslo
+4791880522
http://dhis2.org


Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp

Hi to all,

Dear DHIS users, do anyone knows where we find options for whether to use custom forms,Default or section forms. Previous versions this used to appear on the data entry screen.

···

Muhire Andrew

HMIS/Ministry of Health

First say to yourself what you would be; and then do what you have to do..


From: Knut Staring knutst@gmail.com
To: Muhire Andrew muhireandrew@yahoo.com
Cc:dhis2-users@lists.launchpad.netdhis2-users@lists.launchpad.net
Sent: Thursday, January 12, 2012 5:16 AM
Subject: Re: [Dhis2-users] Running integrity checks on datasets with unlinked data element?

This may be yet another good reason to try and move away from custom forms. I think most of your forms could work well as custom forms.

Knut

On Thursday, January 12, 2012, Muhire Andrew wrote:

Dear jason,

i have custom forms!

------------------------------On Wed, Jan 11, 2012 11:54 AM PST Jason Pickering wrote:>Seems like a good idea. Do you have section forms or custom forms?>>>On Wed, Jan 11, 2012 at 9:18 PM, Muhire Andrew <muhireandrew@yahoo.com > wrote:>>>> Hi to all,>> i would like to run integrity check to see the data element that are in dataset that are not linked to the form. Is this possible? Not data element without dataset ( data element in dataset that is not linked to the form). Because this affects data completeness report(based on data values) where by the target column counts all linked and nonlinked data elements which can confuse users. Per now i just use a manual checking.>>>>>>>>>>>>>>>------------------------------On Wed, Jan 11, 2012 2:06 AM PST Ola Hodne Titlestad wrote:>I would suggest to capture service availability and stock-outs as separate>data elements or even orgunit groups for services, and not as some>interpretation of a 0. Then there is no

need to capture zeros.>Once the data is aggregated up one level all this zero/blank interpretation>is impossible anyway. The complete button also gives an indication of>wheter data is missing of not.>Ola>On Jan 11, 2012 9:45 AM, “Knut Staring” <knutst@gmail.com > wrote:> Here in India, the activities of Subcenters differ quite a bit in the> particular services they offer. Therefore, since all use a standard format,> there will almost always be a number of fields that are not applicable.> Because of this, they maintain that 0 is signficant, because it means “Have> the service, but no cases this month”, whereas a blank field means “Don’t> have the service” in other words N/A.>>>> Sometimes it can even mean something in-between, like “We normally> distribute Iron tablets, but sometimes we don’t have them (blank), and> sometimes we have them but there is no demand (0)”. In my view the latter> semantics is quite complicated, and hard to get consistent, but

I’m being> told the health workers handle it well. Thus, a typical report can consist> of 50 positive integers, 7 blanks and 20 zeros.> Since data entry on the mobile is quite cumbersome and zeros also tend to> fill up the database, I would have preferred zeros to be non-significant> and just go with blanks. But then we would lose the above distinction.> Knut>> → Knut Staring> Dept. of Informatics, University of Oslo> +4791880522> http://dhis2.org>> _______________________________________________> Mailing list: https://launchpad.net/~dhis2-devs> Post to : dhis2-devs@lists.launchpad.net> Unsubscribe : https://launchpad.net/~dhis2-devs > More help > :>> https://help.launchpad.net/ListHelp>>>>> _______________________________________________>> Mailing list: https://launchpad.net/~dhis2-users>> Post to : dhis2-users@lists.launchpad.net>> Unsubscribe : https://launchpad.net/~dhis2-users>> More help : https://help.launchpad.net/ListHelp


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp


Knut Staring

Dept. of Informatics, University of Oslo

+4791880522