Data Approval Workflow - any cautions before I begin?

I am interested in building a data approval workflow for an aggregate data set that already has 12 months of data submitted. The main reason I need an approval workflow is to prevent unapproved data from showing in the pivot tables. What will happen to all of the existing data that was entered before the approval workflow existed? I assume it is, by default, “unapproved,” and it will not show in the pivot tables after this option is selected in the pivot table settings.

Is it going to be difficult to bulk-approve all of these aggregate data values? What is the best approach here?

2 Likes

I have the same question, Mike. Look forward to see if the community has any pointers on this.

1 Like

Hi Mike and Grace,

Just a word of warning, in case you weren’t already aware: the Data Approvals sub-app has just been removed from DHIS2 (release 2.33), so any approvals you implement now will shortly be obsolete:

As far as I am aware, this feature was removed without any planning or advance notice to the community. :slightly_frowning_face: Two of the national HMIS teams I’m working with have been planning to implement Data Approval workflows, but given this sudden change, have decided to put those plans on hold.

2 Likes

Yes, once you have configured data to be approved, and hidden in analytics, then any past, unapproved data will be hidden the next time you rebuild analytics.

Note that when you define an approval workflow, you can choose the period type. So for example, even if your data set is monthly, you could approve quarterly. This means you wouldn’t see the data in analytics until you approve the whole quarter, but it means you have to make fewer approvals.

Also note that you can choose the level(s) for an approval workflow. So if you approve only at the district level, you won’t see any data from the district until you approve the whole district. But it also means that you have to make fewer approvals. At the extreme, if all you want to do is hide the data for a period until you are ready, you could define the approval level at the root (lowest-number) organisation unit level.

If you need to approve past period data in bulk, and you still need many approvals for different periods and/or organisation units, you can try using the Web API. You can see what Web API calls are needed by reading the documentation here here or by approving something through the front end (pre-2.33) and looking at the http calls (for example in the Chrome browser select View → Developer → Developer Tools.) You can execute the Web API calls using curl, or your favorite programming/scripting language.

1 Like

Hi Samuel,

This is quite alarming. I am trying to set up data approvals so that our donors have direct login access to data ASAP. I haven’t uploaded to 2.33 yet, i’m on 2.30, and this news makes me happy about that. Still I’m hesitant to set up something in an app they’ve removed and will re-release.

Are there any work arounds to this? I’m thinking teams could fill in their data and then have someone else log in with data entry approvals and “approve” it before it’s “completed.” This seems really arcane and unfriendly to users…

Any ideas would be appreciated since we are on a tight timeline.

Best,
Grace

1 Like

Hi Jim,

Thanks for this heads up about not being able to see all old (unapproved) data. Have you figured out a way to restrict a person’s approval access to certain OU’s at the same level and/or different attributes/projects? From what I see, I’d need to create an approval level linked to the project (attribute group option sets) and then create a workflow for each project-specific approval level, which seems pretty tedious. Am I missing something?

Cheers,
Grace

1 Like

Thanks @Jim_Grace for such a detailed response. I’m running v2.30, but the documentation is slightly different than the menu options that I am seeing. The documentation references:

System Approval Settings page, you may select the option Hide unapproved data in analytics to hide unapproved data in reports, pivot table, data visualizer and GIS.

But in the System Analytics settings, I see the option:

Max number of years to hide data in analytics

I would like to start with this new data approval workflow for 2020 data only. So if I change this selection from “Never check approval” to “This Year Only,”

  1. If I activate this feature today, should I expect that all 2019 data that is currently entered, and 2019 data that is entered in the future, will be visible in Pivot Tables?
  2. Will it will be acceptable that within the same dataSet, I have some users entering data at orgUnit level 4 (facility), some at OrgUnit Level 3 (district) and others at Level 2 (country)? I want to set Approval to happen only at Level 2, so that no matter where data is entered, it is approved by an officer at the country level, and then visible in the Pivot Tables at the “global” level.
  3. What happens to data that is modified by a superuser after it is approved? Does it get processed in analytics and posted to the Pivot Table, or does it need to be approved a second time?

Thank you so much for your guidance on this.

1 Like

Hi @gawantang,

Users can approve data only for organisation units to which they are assigned. For example, in the Sierra Leone demo DHIS 2 Demo - Sierra Leone, if a user is assigned to the ‘Ngelehun CHC’ facility and has the ‘Approve Data’ authority, they can only approve data for that facility. If they are assigned to the ‘Badjia’ Chiefdom, they can only approve data for that chiefdom and/or facilities within that chiefdom. And so on.

Using sharing, you can restrict the attribute options that a user can see. For example, in the Sierra Leone demo, the ‘ART monthly summary’ dataset is assigned a category combination of ‘Implementing Partner and Projects’. If you use sharing to share only one (or more) implementing partners with a user, and they have the ‘Approve Data’ authority, then they can only approve data for those partners who are shared with them. Alternatively, if you use sharing to share only one (or more) projects with a user, then they can only approve data for those projects that are shared with them.

The PEPFAR implementation of DHIS 2 makes use of both of these restrictions. Users who approve data are typically constrained both by the organisation units to which they are assigned, and also the category options that are shared with them. (The only users who are not so constrained are privileged top-level users who have the ability to correct any data.)

Cheers,
Jim

1 Like

Hi @Mike_Johnson,

First of all, thank you for pointing out that the documentation is out of date. It describes the way the system settings used to work many versions ago, but we forgot to change the documentation when more flexibility was added to the system setting. I am updating the documentation now, and it should reflect the UI in the near future when my documentation changes are reviewed and committed.

  1. Yes, “This year only” means that unapproved 2020 data would be hidden, but all 2019 and previous data would be shown, whether approved or not (and regardless of when it is entered). Don’t forget to change the setting next year to “Last 2 years”, and the year after to “Last 3 years”, etc.! The real reason that this feature was implemented is so that implementations that use approvals for many years could delete database approval records from the database for past years so they don’t clutter up the database forever. (This approval record deletion is not supported by DHIS 2, but could be done directly in the database.)

  2. Yes, if you define a workflow for Level 2, then it will approve all data at level 2 and below, regardless of what level it was entered for.

  3. Data does not have to be approved again to show up in analytics. This is because data approval is not applied to every individual piece of data. Rather, a single approval record in the database indicates that all data is “approved” for a certain combination of workflow/approvalLevel(or below)/period(or contained period)/attributeOptionCombo(if applicable). The same approval record can be used to limit what data can be seen in analytics, depending on the system setting for that. But each database approval record applies to a whole combination of data dimensions, not individual data values.

One source of additional information on data approvals is a talk I recorded at approvals, etc. - YouTube.

Cheers,
Jim

2 Likes

Hi @Jim_Grace and @SamuelJohnson, I’m resurrecting this thread on approvals to ask if there has been a decision on how this workflow will be handled in future versions. I held off on implementing this feature in my system because of your notification that the sub-app would be removed from 2.33, but I never circled back to ask if this would be replaced with a newer and better app or feature?

Hi @Mike_Johnson,

In 2.34 there is a new “Data Approval” app. This brings back the same functionality that was previously in the Data Approval section of the Reports app in versions 2.32 and earlier. We have had several discussions during recent development cycles about how to improve the user interface for approving data, but unfortunately have not yet settled on a new design. So we brought back the previous functionality at least for now.

Cheers,
Jim

Hi Jim,

What authorities do users need in order to create Approval Levels and Approval Workflows? It doesn’t seem like any of the “system” authorities like “Approve Data” allow the creation of the approvals metadata.

Hi @chase.freeman,

You are right. In order to manage approval levels and approval workflows, you need the “Change system settings” authority.

Cheers,
Jim

1 Like

Thanks for your insight, @Jim_Grace