Event data values are stored with option code which is not unique

Hello DHIS Devs,

In DHIS, event capture app while registering an event, we have noticed that the event dataValues (for the dataElements with optionSet) are posted with the selected option code. Similarly, in the event list the event data values are mapped with the option code to show the selected option.

We see a small problem with this approach, since, option code in option is not a unique value, which means there is a possibility of two options having the same option code leading to ambiguity in finding the selected option.

Instead, we think it should be either of these two:
- In Event capture, event data values should be saved with some unique option property (like option id) instead of option code. But, this would be a very huge migration, since, all the existing event data values should be changed from option code to option id.
- In Option, code should be made unique and there should be validations in option creation form to ensure option code is unique. This would be the simpler change to implement and our preferred solution to this issue.

Thanks,
Jhansi

Hi Jhansi,

Your second suggestion is a good one with slight modification: introduce a setting for the option set to enforce unique option codes. This makes it optional per option set. The reason for this is that there are use cases where the option code is used as a value for example If the option set are possible answers to a question. The right answer (option) may have a code of 1 while all the alternative options will be assigned 0. In this case it is not essential to distinguish between the options just that one option was right while the rest were wrong.

If the devs go ahead with your second suggestion, they will be introducing another problem. So making uniqueness of option codes optional would provide a more robust solution and flexibility.

Cheers.

···

**
Busoye Anifalaje (PhD)**
Director of Services (Principal), BAO Systems

UK: +44 7901-740-757 | US: +1 682-307-0986|

busoye@baosystems.com | http://www.baosystems.com |

Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007

On 6 May 2017, at 09:47, jhansirk jhansirk@thoughtworks.com wrote:

Hello DHIS Devs,

In DHIS, event capture app while registering an event, we have noticed that the event dataValues (for the dataElements with optionSet) are posted with the selected option code. Similarly, in the event list the event data values are mapped with the option code to show the selected option.

We see a small problem with this approach, since, option code in option is not a unique value, which means there is a possibility of two options having the same option code leading to ambiguity in finding the selected option.

Instead, we think it should be either of these two:

  • In Event capture, event data values should be saved with some unique option property (like option id) instead of option code. But, this would be a very huge migration, since, all the existing event data values should be changed from option code to option id.
  • In Option, code should be made unique and there should be validations in option creation form to ensure option code is unique. This would be the simpler change to implement and our preferred solution to this issue.

Thanks,
Jhansi


Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help : https://help.launchpad.net/ListHelp

I am not sure what the source of the ambiguity actually is. A given data element is linked to a single option set (at least currently). Since every data value is tied to a data element and the data element is in turn linked to an option set, I am not sure what the source of the ambiguity is?

I think there are good analytical reasons not to make this unique. You may want to have an option set like

Option = “Yes”, “No”

Code = “1”, “0”

Option “TRUE”, “FALSE”

Code = “1”, “0”

From the current analytics, it makes things easier when it comes to aggregation but might make more sense from data entry perspective to have both option sets.

I would suggest you setup your own restriction on this in your database by adding an SQL integrity check which you can use to identify and test against. Maybe something like:

sierra-leone=# SELECT code,COUNT() FROM optionset GROUP BY code HAVING COUNT() > 1;

code | count

image

···

On Sat, May 6, 2017 at 10:57 AM, Busoye Anifalaje busoye@baosystems.com wrote:

Hi Jhansi,

Your second suggestion is a good one with slight modification: introduce a setting for the option set to enforce unique option codes. This makes it optional per option set. The reason for this is that there are use cases where the option code is used as a value for example If the option set are possible answers to a question. The right answer (option) may have a code of 1 while all the alternative options will be assigned 0. In this case it is not essential to distinguish between the options just that one option was right while the rest were wrong.

If the devs go ahead with your second suggestion, they will be introducing another problem. So making uniqueness of option codes optional would provide a more robust solution and flexibility.

Cheers.

**
Busoye Anifalaje (PhD)**
Director of Services (Principal), BAO Systems

UK: +44 7901-740-757 | US: +1 682-307-0986|

busoye@baosystems.com | http://www.baosystems.com |

Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007

On 6 May 2017, at 09:47, jhansirk jhansirk@thoughtworks.com wrote:

Hello DHIS Devs,

In DHIS, event capture app while registering an event, we have noticed that the event dataValues (for the dataElements with optionSet) are posted with the selected option code. Similarly, in the event list the event data values are mapped with the option code to show the selected option.

We see a small problem with this approach, since, option code in option is not a unique value, which means there is a possibility of two options having the same option code leading to ambiguity in finding the selected option.

Instead, we think it should be either of these two:

  • In Event capture, event data values should be saved with some unique option property (like option id) instead of option code. But, this would be a very huge migration, since, all the existing event data values should be changed from option code to option id.
  • In Option, code should be made unique and there should be validations in option creation form to ensure option code is unique. This would be the simpler change to implement and our preferred solution to this issue.

Thanks,
Jhansi


Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help : https://help.launchpad.net/ListHelp


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+46764147049

Hi,

I might have misunderstood both the issue and your example, Jason, but my understanding was that the problem here is that duplicate codes within an option set is allowed, while your example refer to options having the same codes in different option sets?

I would in any case argue for having at least an option for checking for uniqueness of codes within an option set, so that the selections from data entry is stored. I actually thought that was the case already (I think it was at some point?), but noticed last week that it isn’t in 2.26 at least, and I’ve been meaning to write to the list for clarification.

Olav

···

On Sat, May 6, 2017 at 10:57 AM, Busoye Anifalaje busoye@baosystems.com wrote:

Hi Jhansi,

Your second suggestion is a good one with slight modification: introduce a setting for the option set to enforce unique option codes. This makes it optional per option set. The reason for this is that there are use cases where the option code is used as a value for example If the option set are possible answers to a question. The right answer (option) may have a code of 1 while all the alternative options will be assigned 0. In this case it is not essential to distinguish between the options just that one option was right while the rest were wrong.

If the devs go ahead with your second suggestion, they will be introducing another problem. So making uniqueness of option codes optional would provide a more robust solution and flexibility.

Cheers.

<Screen Shot 2017-03-01 at 15.10.20.png>
**
Busoye Anifalaje (PhD)**
Director of Services (Principal), BAO Systems

UK: +44 7901-740-757 | US: +1 682-307-0986|

busoye@baosystems.com | http://www.baosystems.com |

Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007

On 6 May 2017, at 09:47, jhansirk jhansirk@thoughtworks.com wrote:

Hello DHIS Devs,

In DHIS, event capture app while registering an event, we have noticed that the event dataValues (for the dataElements with optionSet) are posted with the selected option code. Similarly, in the event list the event data values are mapped with the option code to show the selected option.

We see a small problem with this approach, since, option code in option is not a unique value, which means there is a possibility of two options having the same option code leading to ambiguity in finding the selected option.

Instead, we think it should be either of these two:

  • In Event capture, event data values should be saved with some unique option property (like option id) instead of option code. But, this would be a very huge migration, since, all the existing event data values should be changed from option code to option id.
  • In Option, code should be made unique and there should be validations in option creation form to ensure option code is unique. This would be the simpler change to implement and our preferred solution to this issue.

Thanks,
Jhansi


Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help : https://help.launchpad.net/ListHelp


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+46764147049

Hi all,

Yes Olav is correct; our concern is that it is possible to assign duplicate codes to multiple options within an optionSet. It presents a data integrity issue when displaying an event because you are not able to determine the specific option selected as the value for a data element. It also poses a usability issue because upon realising the duplicate option codes you aren’t able to go back and edit the option’s code attribute.

Adding an opt-out (or opt-in) attribute for optionSets to require unique codes for each option would be one approach. Another approach could also be providing a warning on the form when a user has entered a code already in use.

Thoughts?

Cheers,

···

On Sun, May 7, 2017 at 11:22 AM, Olav Poppe olav.poppe@me.com wrote:

Hi,
I might have misunderstood both the issue and your example, Jason, but my understanding was that the problem here is that duplicate codes within an option set is allowed, while your example refer to options having the same codes in different option sets?

I would in any case argue for having at least an option for checking for uniqueness of codes within an option set, so that the selections from data entry is stored. I actually thought that was the case already (I think it was at some point?), but noticed last week that it isn’t in 2.26 at least, and I’ve been meaning to write to the list for clarification.

Olav

  1. mai 2017 kl. 11.33 skrev Jason Pickering jason.p.pickering@gmail.com:

I am not sure what the source of the ambiguity actually is. A given data element is linked to a single option set (at least currently). Since every data value is tied to a data element and the data element is in turn linked to an option set, I am not sure what the source of the ambiguity is?

I think there are good analytical reasons not to make this unique. You may want to have an option set like

Option = “Yes”, “No”

Code = “1”, “0”

Option “TRUE”, “FALSE”

Code = “1”, “0”

From the current analytics, it makes things easier when it comes to aggregation but might make more sense from data entry perspective to have both option sets.

I would suggest you setup your own restriction on this in your database by adding an SQL integrity check which you can use to identify and test against. Maybe something like:

sierra-leone=# SELECT code,COUNT() FROM optionset GROUP BY code HAVING COUNT() > 1;

code | count

------±------

| 114

(1 row)

Regards,

Jason


Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help : https://help.launchpad.net/ListHelp

-doh

On Sat, May 6, 2017 at 10:57 AM, Busoye Anifalaje busoye@baosystems.com wrote:

Hi Jhansi,

Your second suggestion is a good one with slight modification: introduce a setting for the option set to enforce unique option codes. This makes it optional per option set. The reason for this is that there are use cases where the option code is used as a value for example If the option set are possible answers to a question. The right answer (option) may have a code of 1 while all the alternative options will be assigned 0. In this case it is not essential to distinguish between the options just that one option was right while the rest were wrong.

If the devs go ahead with your second suggestion, they will be introducing another problem. So making uniqueness of option codes optional would provide a more robust solution and flexibility.

Cheers.

<Screen Shot 2017-03-01 at 15.10.20.png>
**
Busoye Anifalaje (PhD)**
Director of Services (Principal), BAO Systems

UK: +44 7901-740-757 | US: +1 682-307-0986|

busoye@baosystems.com | http://www.baosystems.com |

Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007

On 6 May 2017, at 09:47, jhansirk jhansirk@thoughtworks.com wrote:

Hello DHIS Devs,

In DHIS, event capture app while registering an event, we have noticed that the event dataValues (for the dataElements with optionSet) are posted with the selected option code. Similarly, in the event list the event data values are mapped with the option code to show the selected option.

We see a small problem with this approach, since, option code in option is not a unique value, which means there is a possibility of two options having the same option code leading to ambiguity in finding the selected option.

Instead, we think it should be either of these two:

  • In Event capture, event data values should be saved with some unique option property (like option id) instead of option code. But, this would be a very huge migration, since, all the existing event data values should be changed from option code to option id.
  • In Option, code should be made unique and there should be validations in option creation form to ensure option code is unique. This would be the simpler change to implement and our preferred solution to this issue.

Thanks,
Jhansi


Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help : https://help.launchpad.net/ListHelp


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp


Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+46764147049