Use of Periods

Uwe_Wahser · 3 December 2015 13:31

Dear all,

I understand that the period-dimension for data entry and analysis traditionally
comes from the reporting periods, along which routine summary reports are
entered into the system.

In our setting, where we want to load aggregated data from relational data base
systems into DHIS2 data-sets, we don't really have reporting periods as such
(reporting frequency is daily). Nevertheless there are number of dates to be
respected when reporting data-elements.

Since the period dimension seems to be the only way to represent dates in the
DHIS2 aggregate level, I can think of two options to model dates into a
data-set:

1 - preferred: use the standard period-dimension and code the data-elements
according to a reference date into the period dimension (e.g. Number of Births
-> Date of Birth, Number of registrations -> date of registration) The problem
here is, that I can't seem to find a way of analyzing data without restriction
to a date (e.g. to get the total amount of people based registrations from up to
40 years ago) in pivot table, visualizer or dashboards. The maximum I can get
out is over a time-span of 10 years as DHIS2 forcefully requires the user to
chose a specific period.

2 - use the categories (year, month, day) for each date (e.g BirthYear,
BirthMonth, BirthDay) combined into CategoryCombos and code the dates as
CategoryOptions (inflating of course the number of OptionCombos) although this
seems to be quite flexible, I will be missing out on the dynamic featrues (e.g
current year, last 12 months) and iI might confuse the user as there will be a
date-dimension which is not being used.

How are others using the date dimension when having to aggregate across
unlimited periods WITHIN DHIS2? Is there maybe a setting for the analysis
module, where I can extend the number of years?

Thanks for sharing your experiences,

Uwe

Lars · 3 December 2015 14:07

Hi Uwe,

to me it sounds like you could use the “daily” period type and map this to daily periods.

The daily period format is yyyyMMdd as described in the docs.

So for each data value record you import (as part of a regular dataValueSet), you can format your date (as e.g. 19850401) and import as routine data.

regards,

Lars

···

On Thu, Dec 3, 2015 at 2:31 PM, Uwe Wahser uwe@wahser.de wrote:

Dear all,

I understand that the period-dimension for data entry and analysis traditionally

comes from the reporting periods, along which routine summary reports are

entered into the system.

In our setting, where we want to load aggregated data from relational data base

systems into DHIS2 data-sets, we don’t really have reporting periods as such

(reporting frequency is daily). Nevertheless there are number of dates to be

respected when reporting data-elements.

Since the period dimension seems to be the only way to represent dates in the

DHIS2 aggregate level, I can think of two options to model dates into a

data-set:

1 - preferred: use the standard period-dimension and code the data-elements

according to a reference date into the period dimension (e.g. Number of Births

→ Date of Birth, Number of registrations → date of registration) The problem

here is, that I can’t seem to find a way of analyzing data without restriction

to a date (e.g. to get the total amount of people based registrations from up to

40 years ago) in pivot table, visualizer or dashboards. The maximum I can get

out is over a time-span of 10 years as DHIS2 forcefully requires the user to

chose a specific period.

2 - use the categories (year, month, day) for each date (e.g BirthYear,

BirthMonth, BirthDay) combined into CategoryCombos and code the dates as

CategoryOptions (inflating of course the number of OptionCombos) although this

seems to be quite flexible, I will be missing out on the dynamic featrues (e.g

current year, last 12 months) and iI might confuse the user as there will be a

date-dimension which is not being used.

How are others using the date dimension when having to aggregate across

unlimited periods WITHIN DHIS2? Is there maybe a setting for the analysis

module, where I can extend the number of years?

Thanks for sharing your experiences,

Uwe

Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp

–

Lars Helge Øverland

Lead developer, DHIS 2

University of Oslo

Skype: larshelgeoverland

http://www.dhis2.org

Uwe_Wahser · 3 December 2015 14:20

Hi Lars,

that's the plan (and it thanks to your quick replies over the last couple of
days that already works).

The problem is not getting the date in - it's getting them out, since I import
dates back from the last millenium (actually back from 8 AD because of data
entry errors). I'd like to analyze them regardless of that date. Analysis tables
for those years were created, but in PivotTables I am only able to select up to
10 Years back.

Thanks, Uwe

···

Lars Helge Øverland <larshelge@gmail.com> hat am 3. Dezember 2015 um 15:07
geschrieben:

Hi Uwe,

to me it sounds like you could use the "daily" period type and map this to
daily periods.

The daily period format is *yyyyMMdd* as described in the docs
<https://www.dhis2.org/doc/snapshot/en/developer/html/ch01s04.html>\.

So for each data value record you import (as part of a regular
dataValueSet), you can format your date (as e.g. 19850401) and import as
routine data.

regards,

Lars

On Thu, Dec 3, 2015 at 2:31 PM, Uwe Wahser <uwe@wahser.de> wrote:

> Dear all,
>
> I understand that the period-dimension for data entry and analysis
> traditionally
> comes from the reporting periods, along which routine summary reports are
> entered into the system.
>
> In our setting, where we want to load aggregated data from relational data
> base
> systems into DHIS2 data-sets, we don't really have reporting periods as
> such
> (reporting frequency is daily). Nevertheless there are number of dates to
> be
> respected when reporting data-elements.
>
> Since the period dimension seems to be the only way to represent dates in
> the
> DHIS2 aggregate level, I can think of two options to model dates into a
> data-set:
>
> 1 - preferred: use the standard period-dimension and code the data-elements
> according to a reference date into the period dimension (e.g. Number of
> Births
> -> Date of Birth, Number of registrations -> date of registration) The
> problem
> here is, that I can't seem to find a way of analyzing data without
> restriction
> to a date (e.g. to get the total amount of people based registrations from
> up to
> 40 years ago) in pivot table, visualizer or dashboards. The maximum I can
> get
> out is over a time-span of 10 years as DHIS2 forcefully requires the user
> to
> chose a specific period.
>
> 2 - use the categories (year, month, day) for each date (e.g BirthYear,
> BirthMonth, BirthDay) combined into CategoryCombos and code the dates as
> CategoryOptions (inflating of course the number of OptionCombos) although
> this
> seems to be quite flexible, I will be missing out on the dynamic featrues
> (e.g
> current year, last 12 months) and iI might confuse the user as there will
> be a
> date-dimension which is not being used.
>
> How are others using the date dimension when having to aggregate across
> unlimited periods WITHIN DHIS2? Is there maybe a setting for the analysis
> module, where I can extend the number of years?
>
> Thanks for sharing your experiences,
>
> Uwe
>
> _______________________________________________
> Mailing list: https://launchpad.net/~dhis2-users
> Post to : dhis2-users@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~dhis2-users
> More help : https://help.launchpad.net/ListHelp
>

--
Lars Helge Øverland
Lead developer, DHIS 2
University of Oslo
Skype: larshelgeoverland
http://www.dhis2.org <https://www.dhis2.org>

Lars · 3 December 2015 14:27

Right. In fact you can click the "Prev year" button in the period section
several times to list any year. You can then select all the years you need,
put period as a filter under layout.

···

On Thu, Dec 3, 2015 at 3:20 PM, Uwe Wahser <uwe@wahser.de> wrote:

Hi Lars,

that's the plan (and it thanks to your quick replies over the last couple
of
days that already works).

The problem is not getting the date in - it's getting them out, since I
import
dates back from the last millenium (actually back from 8 AD because of data
entry errors). I'd like to analyze them regardless of that date. Analysis
tables
for those years were created, but in PivotTables I am only able to select
up to
10 Years back.

Uwe_Wahser · 3 December 2015 14:45

Oh darn, I though those were constants for the last and the next year (while I
just realize that "next year" probably wouldn't make sense )

Ok, that will do for the moment. Still this is a bit painfull if I want to
select all the years back to the oldest birth year - all the other categories
have a check box for selecting all possible values - is there a reason, that
this is not possible for periods? If no, can I file a blueprint?

Regards, Uwe

···

Lars Helge Øverland <larshelge@gmail.com> hat am 3. Dezember 2015 um 15:27
geschrieben:

On Thu, Dec 3, 2015 at 3:20 PM, Uwe Wahser <uwe@wahser.de> wrote:

> Hi Lars,
>
> that's the plan (and it thanks to your quick replies over the last couple
> of
> days that already works).
>
> The problem is not getting the date in - it's getting them out, since I
> import
> dates back from the last millenium (actually back from 8 AD because of data
> entry errors). I'd like to analyze them regardless of that date. Analysis
> tables
> for those years were created, but in PivotTables I am only able to select
> up to
> 10 Years back.
>

Right. In fact you can click the "Prev year" button in the period section
several times to list any year. You can then select all the years you need,
put period as a filter under layout.

[image: Inline image 1]

Lars · 3 December 2015 16:16

Oh darn, I though those were constants for the last and the next year
(while I
just realize that "next year" probably wouldn't make sense )

No prob.

Ok, that will do for the moment. Still this is a bit painfull if I want to
select all the years back to the oldest birth year - all the other
categories
have a check box for selecting all possible values - is there a reason,
that
this is not possible for periods? If no, can I file a blueprint?

Not sure if that makes sense. The other dimensions have a finite set of
dimensional items, whereas the period dimension less so.

I think what we could do is when "Yearly" is selected as period type, we
should the 20 last years, and the "previous" button could jump 20 years
back in time instead of just one. Then you could use the "select all" /
double-arrow button to get the years over. This would greatly reduce the
number of clicks. Would that work?

···

On Thu, Dec 3, 2015 at 3:45 PM, Uwe Wahser <uwe@wahser.de> wrote:

Regards, Uwe

> Lars Helge Øverland <larshelge@gmail.com> hat am 3. Dezember 2015 um
15:27
> geschrieben:
>
>
> On Thu, Dec 3, 2015 at 3:20 PM, Uwe Wahser <uwe@wahser.de> wrote:
>
> > Hi Lars,
> >
> > that's the plan (and it thanks to your quick replies over the last
couple
> > of
> > days that already works).
> >
> > The problem is not getting the date in - it's getting them out, since I
> > import
> > dates back from the last millenium (actually back from 8 AD because of
data
> > entry errors). I'd like to analyze them regardless of that date.
Analysis
> > tables
> > for those years were created, but in PivotTables I am only able to
select
> > up to
> > 10 Years back.
> >
>
> Right. In fact you can click the "Prev year" button in the period section
> several times to list any year. You can then select all the years you
need,
> put period as a filter under layout.
>
> [image: Inline image 1]

--
Lars Helge Øverland
Lead developer, DHIS 2
University of Oslo
Skype: larshelgeoverland
http://www.dhis2.org <https://www.dhis2.org>

Uwe_Wahser · 10 December 2015 21:48

Hi Lars,

sorry for the late reply but here are some more thoughts on the date dimension,

Regards, Uwe

From the point of data analysis it makes sense in the way that I am able to neglect the date dimension for my analysis, just like I am able to neglect any other dimension. Is it really necesarry to have the date dimension as a mandatory filter?

But I can get your point with regard to the technical imlementation (which I don't know) of filters in DHIS2:

- if you implemented filters as inner joins on tables with filter-items or as item lists in the IN(a,b,c) operator of the WHERE clause, I could understand that it is not possible to include an infinite list of periods, when the users checks 'select all'

- if checking 'select all' simply means not having any filter on the respective dimension, an infinite list shouldn't be an issue.

In the same sense: does it make a difference for the other dimensions, if I check 'select all' or if I don't touch that dimension at all for analysis?

That would help a bit, but only if I limit the range of valid dates when importing data by mapping them to a certain min or max date. Still I'll have to trust, that the users are not to lazy to really select all available years.

But generally I feel that it makes sense to look at the period selection again with regard to usability: e.g. sorting doesn't work on the left list after selecting and deselecting more than one batch of periods (e.g. 30 years); when switching from year to quarter, then deselecting the years, the years will be inserted on the left list into the quarters. This is quite confusing for the user and a source of error when having long value lists.

···

          Ok, that will do for the moment. Still this is a bit painfull if I want to

          select all the years back to the oldest birth year - all the other categories

          have a check box for selecting all possible values - is there a reason, that

          this is not possible for periods? If no, can I file a blueprint?

          Not sure if that makes sense. The other dimensions have a finite set of dimensional items, whereas the period dimension less so.

          I think what we could do is when "Yearly" is selected as period type, we should the 20 last years, and the "previous" button could jump 20 years back in time instead of just one. Then you could use the "select all" / double-arrow button to get the years over. This would greatly reduce the number of clicks. Would that work?

Regards, Uwe

          > Lars Helge Øverland <larshelge@gmail.com              > hat am 3. Dezember 2015 um 15:27

          > geschrieben:

              >

              > On Thu, Dec 3, 2015 at 3:20 PM, Uwe Wahser <uwe@wahser.de                  > wrote:

              >

              > > Hi Lars,

              > >

              > > that's the plan (and it thanks to your quick replies over the last couple

              > > of

              > > days that already works).

              > >

              > > The problem is not getting the date in - it's getting them out, since I

              > > import

              > > dates back from the last millenium (actually back from 8 AD because of data

              > > entry errors). I'd like to analyze them regardless of that date. Analysis

              > > tables

              > > for those years were created, but in PivotTables I am only able to select

              > > up to

              > > 10 Years back.

              > >

              >

              > Right. In fact you can click the "Prev year" button in the period section

              > several times to list any year. You can then select all the years you need,

              > put period as a filter under layout.

              >

              > [image: Inline image 1]

–

                                              Lars Helge Øverland

                                            Lead developer, DHIS 2

                                            University of Oslo

                                              Skype: larshelgeoverland

http://www.dhis2.org