Predictor generates 0 when output DE + category option combo equals the sampled input

Support - Assistance technique
1

TL;DR: A predictor can’t seed from a hand-entered value in its own output DE+COC slot - when the output data element + category option combo equals the sampled operand, the previous period reads as empty and the run generates 0. Changing only the output COC fixes it. Is this an intended boundary, and if so what’s the right pattern for “copy previous period into the same DE+COC”? (Confirmed 2.42.3.1, reproduced on Play 2.42.5.1.)

A predictor whose output DE + category option combo (COC) is identical to the operand it samples never reads the previous period’s stored value. The avg()/sum() sample comes back empty, so the run reports “Generated 0 predictions” (or writes 0 under “Never skip”). Changing only the output COC makes it work.

Version: Confirmed on 2.42.3.1 (self-hosted), reproduced on 2.42.5.1 (Play).

I’m trying to copy a value forward one period into the same DE/COC/orgUnit (e.g. carry an “Imprest level” from June into July) with avg(#{X.C}), Monthly, Sequential sample 1, Annual 0, “Skip if all values missing”. So the output slot X.C equals the sampled operand. Expected: July reads June’s stored value. Actual: “Generated 0 predictions”, nothing written (Never skip writes 0, confirming the previous period reads as empty).

Minimal repro on Play (SL demo, stable-2-42-5-1) - two predictors identical except the output COC. Input is ANC 1st visit / Fixed in both; one outputs back to Fixed (self-referential), the other to Outreach.

Same input operand, same seed, same run - the only variable is whether the output COC equals the sampled input COC. No analytics run was needed (control worked immediately), so this isn’t analytics staleness.

Objects: DE fbfJHSPpUQD (ANC 1st visit) · COCs pq2XI5kz2BY (Fixed) / PT59n8BQbqM (Outreach) · OU DiszpKrYNg8 (Ngelehun CHC) · level m9lBJogzE95 (Facility).

Question: Is it expected that a predictor won’t read a pre-existing value in its own output DE+COC slot as a previous-period sample? I understand self-reference works for feeding the predictor’s own generated values forward across a chained multi-period run - but here I’m seeding from an existing hand-entered value and it’s never sampled. If that’s the intended boundary, what’s the recommended pattern for a plain “copy previous period into the same DE+COC”?

2

Hi @Kenyuri

This is discussed in a ticket here “DHIS2-19235 - Predictors overriding value when previous value is null” – note Jim’s comment: “The trick is not to avoid overwriting the value – which is problematic as discussed above – but to tell the expression to write the same value back as the predicted value.”

As @Jim_Grace mentions in the comments, for this to work as expected there are specifics that need to be applied:

The predictor generator expression using the if() function would look like:
if(count(starting value)==0,starting value,sum(starting value + received - disbursed))
This is because in a predictor expression any value within an aggregate function like count() or sum() will aggregate the values over all the sample periods, whereas any value not within an aggregate function will access the value in the predictor output period.
Note that if you use the default missing value strategy Skip if all values are missing this will compute a new starting value if the old starting value is there, whether or not the received value and/or the disbursed value are present (any missing values are replaced with zero).

You’ll also notice in the comments that Jim’s using the full #{DE.COC} notations #{t99PL3gUxIl.HllvX50cXC0} instead of #{t99PL3gUxIl}.

Please go over the comments in the ticket and apply the feedback from Jim, and if you still face an issue, feel free to reply here.

Keep us updated please, thanks! :folded_hands:

3

Hi @Gassim Thanks for the suggestion. I tested the if(count(x)==0, x, avg(x)) approach on my instance (2.42.x) and can confirm it does not resolve the case of a pure carry-forward predictor i.e. one whose generator samples its own output DE + COC, with no other data elements contributing to the result.

To isolate the behaviour, I kept the same operand #{DE.COC} (where output = sampled) and swapped only the aggregate function, predicting June from a May value of 50:

  • count(#{DE.COC}) → 1
  • sum(#{DE.COC}) → 0
  • max(#{DE.COC}) → 0
  • avg(#{DE.COC}) → 0

So count() confirms the May sample is found, but every value-returning aggregate reads it as 0. That means the if(count==0, …) guard doesn’t help in this scenario: the guard passes (count = 1), but the else branch is still avg(self) = 0, so the predictor writes nothing useful.

My reading is that Jim’s workaround relies on the real signal coming from other data elements (e.g. received/disbursed in a stock model), where the aggregates return meaningful values. When the only input is the self-referenced output value itself, the aggregates resolve to 0 and there’s nothing to carry forward.

4

Sharing what worked for me, in case it helps others hitting this.

The if(count(x)==0, …) idea didn’t fix my pure carry-forward case (copying the same DE + COC forward each month). The catch: when the sampled operand is the same DE + COC as the output, DHIS2 finds the sample (count() returns 1) but reads its value as 0 inside sum/avg/max. So any expression that samples itself resolves to 0 - including if(count==0, …), because its else branch is still avg(self).

The fix is to avoid the self-reference entirely: sample the data element at total level, then subtract the other category option combos. What’s left equals your target combo, and none of the operands is the output combo - so nothing gets zeroed:

avg(#{DE})
  - avg(#{DE.otherCombo1})
  - avg(#{DE.otherCombo2})

Missing value strategy: Skip if all values are missing. I tested it both with the other combos populated and with only the target combo present - it carried the correct value forward in both cases.

Two things to watch:

  • It’s tied to your category combo’s exact COC set. If you add a new category option combo later, you’ll need to subtract it here too.
  • A month that has a sibling combo but no target value will produce 0 rather than blank.

Clean predictor-only fix - no extra data elements and no scripts needed. :slightly_smiling_face:

5

Amazing! Thanks for sharing the solution @Kenyuri and I’m glad you figure it out. Nice explanation.. :clap: