I think this is part of a bigger set of issues, namely the temporal aspect of associations of orgunits with orgunit groups and datasets.
For instance, in the completeness analysis, the completeness figures do not take into account whether the orgunit has belonged to a dataset at some point in time, or what the opening date of that orgunit is. Both of these aspects of time really influence the completeness. A facility should not really count as being part of the completeness analysis, unless it has opened, and it should still continue to count as part of the facilities for which constitute the completeness figures for the time periods for which it was a member. It may be removed or closed, but its membership should still count for when it was a member.
Similarly, with orgunit groups. We have attempted to use orgunit groups to facilitate attribution of data values to particular orgunit groups. This is needed particular in PEPFAR related countries, where there is a geographical hierarchy, but also a funding mechanism-project-implementing partner hierarchy. The problem is however that partners often switch facilities. However the values recorded during the point in time when they were a member of a particular orgunit group, should continue to be attributed to that orgunit group (potentially). I say potentially, because these types of groups really do have a temporal dimension. Something similar has come up in one of the countries I am involved in, namely when a particular facility is “upgraded” from a “health center” to a “health post” and one looks at an indicator value by this orgunit group, there should be attribution to the “health center” group for when it was a member of this group up until the point in time which it switches to another group (e.g. the health center).
I am not so sure how feasible it would be to keep the full history of the analytics. It seems horribly complex, as Lars points out, a facility may be incorrectly created in place in the hierarchy, stay there for a week, and then be moved. This history should never be kept. However, keeping a snapshot of the analytics at a particular point in time, may be the better answer.
I am not sure exactly how all of this needs to happen, but I am sure we need something to deal with some of these issues. I personally think having some mechanism to allow temporal membership in an orgunit group would be a good start, and allow analysis through this view of reality, as opposed to the current snapshot view which is the only thing we can use as the moment.
Regards,
Jason
···
On Mon, Oct 21, 2013 at 4:49 PM, Lars Helge Øverland larshelge@gmail.com wrote:
Hi Alvin,
its a good point and something we have been thinking about for a while. This is something that we definitely need to act on soon.
The challenge is to find a sensible level of detail for what to store, as there are costs in terms of i) system complexity and ii) aggregation performance degradation involved here. For example, if facilities are moved around in administrative boundaries every month, how do you calculate the yearly total for the whole province?
My suggestion, as a reasonable trade-off, would be to create snapshots of the organisation unit hierarchy at the end of each year. These snapshots are then used for analytics queries for past data. So for instance if you query today for “last 12 months”, it would use the current “live” organisation unit hierarchy for months Jan-Sept 2013, then the “2012 snapshot” hierarchy for months Oct-Dec 2012.
On an implementation note, this will fit well the analytics solution where data is partitioned across years - each data partition will have a corresponding org unit hierarchy, making aggregation simple.
Lars
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help : https://help.launchpad.net/ListHelp