We recently upgraded to v42 and had a few program indicators stop working
Previously, we had a program indicator that counted days between the incident date and a date type data element in one of two non-repeatable program stages. The PI was set up using the event analytics type with an expression of d2:daysBetween(V{incident_date},#{stage_uid.de_uid}) and a filter of d2:hasValue(#{stage_uid.de_uid}).
In version 42.4, the same PI set up is now calculating as 0 for most periods.
I’m able to re-create the PI as an enrollment type PI but I would like to avoid this if possible as these types of PI typically do not perform well in data viz apps.
I’m attaching the various PI configurations I’ve tried in troubleshooting the issue here and posted the results in the 2 versions below. programIndicators_sharing.json (4.4 KB)
Version 2.42.4
Version 2.37.8
Any insights from the @tracker-analytics team would be great, especially if there is a solution without using enrollment type indicators (e.g. test PI 5) as these always take time to load or end up timing out depending on the report set up
Thank you very much for this @Kris_Reinhardt - the team is quite busy with the last weeks before 2.43 code freeze but if you could create a JIRA issue I’ll ask them to have a look at this soon.
Thanks @brian for taking a look. I had also noticed that the indicator was working as expected in the play demo
The orgUnitField is null in the the test PI that I shared. I also tested enrollment and registering org unit but got the same results. In our usecase, orgUnits should not change between TEI, enrollment, and events.
Is there something we can check to see if there was issue with the flyway script?
Since v40, the PI orgUnitField should be not null. In >90% of cases, an event analytics type program indicator should have orgUnitField set to event organisation unit (default).
Could you try updating that in your PI, and checking that it works?
I think that for DHIS2 version upgrades, the orgunitfield should be update to the default for the PI type, and not to null. If that seems to be the issue here, could you please make a Jira ticket for this? Ideal if you can specify the version where it orgUnitField was dropped (v40?) but I know it takes time to verify.
Sorry @brian, I wasn’t clear. The org unit field in the UI is present and seems to be event organization unit (default) by default for all my program indicators. In the API the orgUnitField is not present. But this seems to align with the expected behavior based on the JIRA you shared.
I took another look at the PI behavior in my environment and found a couple older cases I hadn’t seen before where my test PI (#1-4) were calculating a value other than 0. I used these cases to try to figure out which dates were being used in the calculation. It turns out the calculation was being made with the event date rather than the incident date. Generally, in our use case, the event date is the same as the data element date I’m using in the calculation (hence why the majority of the PI values are calculating to 0). However, there were a few exceptions where the event date and the data element date were not the same (probably due to a temporary issue in the program rule we use to populate the data element from the event date.
As an example, I found a case where my test 5 PI was correctly calculating to 2. But my test PI 1-4 were calculating to -6. For this case, I can see that the calculation PI 1-4 are making are based on the event date, rather than the incident date.
Because test 5 is the only PI that’s working in your tests, I think the difference may be because:
a) its the only valid PI using the “date type data element” (autofilled by incident date) in analytic period boundaries… or..
b) its the only valid PI calculating the difference between two date-type data element values in expression, with data elements in separate program stages. I’m assuming the second DE is autofilled by incident date?
PI 6 is likely invalid as it included && V{program_stage_id} == 'k66SDYD3nLT' in the filter, and since this is program stage criteria is being checked against the enrolment, then its not valid.
If its b) then this checks out:
That does seem to be a bug if the PI is accessing the event date instead of incident date in the expression. The best way to know is to check the server logs while the PI is being queried, to see how the PI is translated into SQL.
The only workaround I can think of is: 1) create a new PR to autoassign incident date to a data element in program stage k66SDYD3nLT, then 2) export all those past events, and re-import with &skipRuleEngine=false to assign the values, Tracker - DHIS2 Documentation and 3) use an event type PI with d2:daysBetween(#{INCIDENT_DATE_DE},#{k66SDYD3nLT.l4DpMg9f5eG}) in expression.
I’m working on checking the logs so I’ll report back shortly on this. I did a quick check and saw that when I changed the incident date in my PI expression to event date, I got the same result
The workaround proposed will not work for us as the incident date is set to equal the 1st program stage event date and data element in that event. The comparison I want to make is between the incident date and the second stage data element (which is set to be equal to the event date for the second stage). This indicator was originally set up prior to enrollment type PI and I would like to keep it that way as enrollment type PI take a long time to load in reports.
Hi @brian, the underlying SQL for the Days between - Test 2 indicator is:
dhis2=> select sum((cast(case when ax."ps" = 'k66SDYD3nLT' then "l4DpMg9f5eG" else null end as date) - cast(occurreddate as date))) as value,ax."yearly" from analytics_event_pkijnrcfmur as ax where (ax."yearly" in ('2025', '2026') ) and ax."uidlevel1" in ('a2cf79e8f13') and ((case when ax."ps" = 'k66SDYD3nLT' then "l4DpMg9f5eG" else null end is not null)) and ("l4DpMg9f5eG" is not null) and ax."yearly" in ('2026', '2025') group by ax."yearly";
value | yearly
-------+--------
10 | 2025
0 | 2026
(2 rows)
From my reading of the SQL it looks like it’s only using the occurreddate for the program stage of the comparison data element, which makes me think it’s using the event date in the calculation rather than the incident date given that the event date is now referred to as the occurreddate with the underlying changes on the tracker side made in more recent versions.
