I defined a new validation rule for “Total number of yellow fever cases”
Left side was a sum of all yellow fever cases (OPD, IP discharge, deaths) dis-aggregated by three age groups each (Total of nine data elements).
Right side was defined as a constant (1.0). “Skip for missing values” was unchecked for both sides. Operator was >=. Period type was set to monthly (Same as the data collection frequency). Other options (Sequential sample count, etc) were left blank.
I run the validation rule analysis and get violations basically everywhere
Organisation unit
Period
Importance
Left side description
Value
Operator
Value
Right side description
Details
ce Foo Orgunit
September 2013
Medium
Total yellow fever cases
0.0
=
1.0
Zero
… (many more here)
What I do not understand is why this rule is being tripped. For this data element, (Yellow fever) we are not recording zeros, and the values therefore should always be NULL, unless there happens to be a value entered greater than zero.
Maybe again, this is a misunderstanding on my part of how the validation rule analysis handles blank values. In this case, I would expect that if “Skip for missing values” is not checked, that the NULLS would be handled effectively as zeros, so that when the multiple data elements are summed up, the left hand side is zero. It seems that is not the case, or else I am missing something.
This use case is quite common for disease surveillance, whereby data elements will typically be NULL (or perhaps zero), but most likely null, since the values are almost always zero, and it does not make sense to record anything other than non-zeros here.
Yeah, I see what you mean. Somehow it seems confusing for some reason.
However, I still cannot get it to work. In most cases, these “Yellow fever” cases are data entry errors, and this is why we need to try and keep an eye on them in particular. So, I found a facility where I know the value is not equal to zero. I altered the data validation rule so that it only included a single data element, where I know there is data not equal to zero. Regardless of the combinations which I tried (<, >, ==, !=, etc) I could not get it to work.
I think there is also something strange with the rule creation dialog, perhaps a bug. When I create a surveillance rule the first time, I see a different dialog for the rule. I press save, and it looks fine. I attempt to do an analysis (no hits yet) and then I go back and edit the rule. When I press save the second time, I now see the “Validation” dialog, and not the “Surveillance” one. The “Rule type” has not apparently changed to “Validation” from “Surveillance”. Was able to reproduce this on the demo site.
Will keep trying, but I think we need to fine-tune the documentation a bit (as well as test the functionality) as I am not getting the result I expect.
For surveillance rules, like other validation rules, the equation effectively tests for “the usual conditions.” When FALSE, an alert is generated. When TRUE no alert is generated. (Maybe we could improve the documentation about this!) So if you want to see an alert when yellow fever cases are >= 1, then you define the equation as yellow fever cases < 1.
You are right that ‘if “Skip for missing values” is not checked, that the NULLS would be handled effectively as zeros’. This seems to be why the rules are showing up with your NULL yellow fever case counts reporting as zero. The main problem is that you were using “>= 1.0” (when you want to see the alert) instead of “< 1.0” (when you don’t want to see the alert.)
If the yellow fever cases count is a single data element value, and you are testing for “< 1.0”, then it shouldn’t matter whether “Skip for missing values” is checked or unchecked. If checked, the rule will be skipped on a NULL value, so no alert will be generated. If unchecked, a NULL is converted to 0.0, which is less than 1.0, so no alert is generated in this case either.
However if the left side of the equation adds up yellow fever cases from several data element values (like yellow fever counts by age group), then it might matter whether “Skip for missing values” is checked or not. If “Skip for missing values” is checked and any value is missing, the rule is skipped. But if “Skip for missing values” is unchecked, then any missing values are replaced with zeros. This could be useful if one age group count is NULL but another age group count is 1. It could still trigger an alert.
I defined a new validation rule for “Total number of yellow fever cases”
Left side was a sum of all yellow fever cases (OPD, IP discharge, deaths) dis-aggregated by three age groups each (Total of nine data elements).
Right side was defined as a constant (1.0). “Skip for missing values” was unchecked for both sides. Operator was >=. Period type was set to monthly (Same as the data collection frequency). Other options (Sequential sample count, etc) were left blank.
I run the validation rule analysis and get violations basically everywhere
Organisation unit
Period
Importance
Left side description
Value
Operator
Value
Right side description
Details
ce Foo Orgunit
September 2013
Medium
Total yellow fever cases
0.0
=
1.0
Zero
… (many more here)
What I do not understand is why this rule is being tripped. For this data element, (Yellow fever) we are not recording zeros, and the values therefore should always be NULL, unless there happens to be a value entered greater than zero.
Maybe again, this is a misunderstanding on my part of how the validation rule analysis handles blank values. In this case, I would expect that if “Skip for missing values” is not checked, that the NULLS would be handled effectively as zeros, so that when the multiple data elements are summed up, the left hand side is zero. It seems that is not the case, or else I am missing something.
This use case is quite common for disease surveillance, whereby data elements will typically be NULL (or perhaps zero), but most likely null, since the values are almost always zero, and it does not make sense to record anything other than non-zeros here.
Hi Lars,
I have not managed to figure out /implement the surveillance rules yet, but the normal validation rules seem to be working as expected now. Thanks.
Best regards,
Jason
···
On Fri, Nov 29, 2013 at 5:09 PM, Lars Helge Øverland larshelge@gmail.com wrote:
Hi Jason,
for the record, we did a bug-fix related to validation rules some time ago, is this now okay?
Hi Lars,
I have not managed to figure out /implement the surveillance rules yet, but the normal validation rules seem to be working as expected now. Thanks.
Best regards,
Jason
On Fri, Nov 29, 2013 at 5:09 PM, Lars Helge Øverland larshelge@gmail.com wrote:
Hi Jason,
for the record, we did a bug-fix related to validation rules some time ago, is this now okay?
The following explains a bit more about surveillance rules and how they are similar to, and different from, other validation rules. We might also consider whether/how to distribute some of these ideas more widely. Some of these ideas might go into future user documentation. We might also consider putting some of this kind of information into a future topical DHIS forum, if we want to explain things in more depth without making the user documentation too huge.
Cheers,
Jim
···
On Fri, Nov 29, 2013 at 10:18 AM, Lars Helge Øverland larshelge@gmail.com wrote:
Okay thanks. Let us know if you have any questions of if something particular is confusing.
Hi Lars,
I have not managed to figure out /implement the surveillance rules yet, but the normal validation rules seem to be working as expected now. Thanks.
Best regards,
Jason
On Fri, Nov 29, 2013 at 5:09 PM, Lars Helge Øverland larshelge@gmail.com wrote:
Hi Jason,
for the record, we did a bug-fix related to validation rules some time ago, is this now okay?
