We are testing out 2.38.1.1 for a possible upgrade in a few months time. One of the largest issues we’ve seen so far is with the added feature of validating program rules, and having this feature prevent data to be saved in the server if errors exist.
Most notably so far is another bug that is exacerbated by the validation: When saving, the System seems to take 2 digit numbers (i.e. 12, 13) to be less than 1 digit numbers (i.e 2,3). Presumably this is either taking the first digit only, or doing some kind of an alphabetical sorting of the two values to see which is coming first; not sure. In any event, on the front end, this error does not occur, but when looking at the console, the error is the given reason for why an event can’t be saved.
In the image below, the An. gambiae s.l.: Number collected (value=12) and the An. gambiae s.l.: Number dissected (Value=2) are not being shown as errors in entry, but are coming up as the program rule response for why the data could not be synced. On the right side of the screen is the program rule relevant (simply, if collected < dissected, show an error).
I’m going to work on replicating this in Play immediately. But my thought as well is that there are quite a number of issues with this new feature, and thought it might be useful to list them all out/hear if people have workarounds for them already.
Because Validations are being made for syncing abilities:
This will limit data entered into a system where either program rules weren’t present previously (and it wouldn’t have come up as an error) or data was hidden and thus the error wasn’t preventing the syncing (i.e. from a previous version of DHIS2)
The limiting of syncing not only disables the ability to sync, but it completely wipes out the data from being able to be edited so that it can be synced. The only way to discover the issue is to open the console and track down the program Rule(s) that are preventing the syncing, and no matter what you’d have to re-enter all data. The error message that appears is very bland “could not sync event” and could easily be misinterpreted as a system error/internet connection error/etc.
Already mentioned, but these errors are being validated on DEs that are also not being shown in the capture app. I.e. if a Data Element is being hidden, and there is an error that is appearing on the Data Element, the Capture app will not prevent you from trying to sync, but the system will prevent you from syncing (and will result in you losing all progress)
Already mentioend, but these errors are also using different algorithms to determine what is and is not an error. This is at least true for 3. above, and for numbers with greater than 1 digit against another number…it feels like it could be the tip of the iceberg in terms of issues that could be found over time.
I’d very much like to propose having an option to disable the prevention of data syncing based on program rules, both as an option in the DHIS2 Settings App, but also as a toggleable selection in Import/Export app. Welcome to other ideas or clarifications/workarounds/corrections to any/all what I’ve brought above!
Same Problem in Play. Cloned hemologlobin value Data Element, put it in the Antenatal Care Visit Program, gave it a Program Rule Variable, and created a program rule that should error iff hemoglobin is less than the cloned hemoglobin.
This rule does not fire in the capture app (i.e. the front end is correct), but does prevent data to be synced because it fires in the backend.
@Gassim would [DHIS2-10802] - Jira be the relevant place to submit this feedback? Interestingly I can’t seem to replicate this issue in 2.37.8 on Play, so I’m not sure if I can track down the specific place where this is taking place. Appreciate any insight here!
Hi @Matthew_Boddie,
as you experienced there is a bug in the evaluation of expressions that only involves Program Rule Variables. Rule-engine is not able to identify the right type of the variable and it is always considering it a String, this is why “9” > “12”.
To make it work with a workaround you can rewrite your expression as
The + 1 - 1 will force rule engine to treat the variable as a number and the expression will be evaluated correctly.
Of course this is just a workaround, we will tackle the Jira issue that you created as soon as possible.
We will also discuss how to improve the usability of the Capture App, as you can see from the response in the payload, there are information about the error that are not shown in the App.
The API is already ready to skip rule engine passing a parameter, so we can also discuss how to enable/disable it from the App.
This error is happening only starting from 2.38 because we switched to new endpoints for tracker that are evaluating more types of program rules compared with the old endpoints.
I hope you are at least unblocked for now.
Cheers,
Enrico
@enrico thanks for this context and for the solution. This is something that we’ve also noticed in Android, and will look to incorporate this fix into both aspects immediately! Back soon.
Update: The +1, -1 does seem to fix the errors for number values being treated as strings. Thanks for this.
Now having these filtered out, we are experiencing different errors whereby d2:hasValue() formula is not being respected on the front end (a bug on the capture side), but is being respected and preventing syncing (correctly, in a fashion) on the back-end. I recognize this is a different scope—but the larger point remains that there are variations on both sides of this coin, and it is producing a very hard to follow inability to sync.
If this is not affecting the same issue with the PRV you discussed in this topic then please it’s recommended to create a new topic. Will make sure to follow up!
Absolutely, will do so—the larger point I’m trying to make is that it should never be the case that I’m allowed through the front end to make it to the attempt to sync, if the backend is going to derail the sync based on a program rule. So just wanted to note that here.
Just as another follow-up @enrico , is it possible that in 2.38.1.1 I could roll back to a previous version of the Capture App specifically which does not have this rule engine parameter enabled?
@phil as it relates to the note on the capture app releases that I asked about on a different topic, hope you don’t mind if I tag you here as well!
Thanks @Gassim These I think are the same options available in 2.38.1.1 in the App Management, and don’t go to a 2.37 capture version. It sounds like I’m missing a larger point though, and that its 2.38.1.1 that has a parameter active which would be active regardless of the Capture App version being used.
