Define week start day for Weekly periods

Johannes_Schnitzler · 29 June 2013 21:36

Dear Randy,

Thank you very much for the reply and suggestion.

The selection of the week will be of course tricky as the week will be different for ISOweeks compared to Cambodian weeks.- at least for some years. In addition I would of like to use all the analysis methods available in DHIS for analysis therefore there would be the need to “link” the analysis modules to the “extra table” with the calculated Year Weeks.

Has anyone tried to change the dates / weeks for data entry. This would at least solve the problem for the user during data entry. The “correct” date would then also be stored in the periods table. This solves of course not the question how can the data be analyzed without the need for custom queries.

Best regards,

Johannes

···

On 29 June 2013 11:09, Wilson,Randy rwilson@msh.org wrote:

Hi Johannes,

I’m not sure how DHIS-2 automatically creates the entries in the period table, but once they are created would one of the following options work?

· Pre-fill the period tables with weekly records for several years forward, then hack the period table to reset the start/end dates for weekly periods using an update query – adding or subtracting days depending upon when you want the start day.

· Create a period translation table (periodid, newstartdate, newenddate) that you can link to the datavalue table and produce your analyses according to the desired time frames.

In Rwanda we end up doing a lot of our analyses with custom SQL queries so option 2 would probably work for us – though users would have to be told to disregard the start date displayed during data entry – and focus on the week number.

Randy

From: Dhis2-users [mailto:dhis2-users-bounces+rwilson=msh.org@lists.launchpad.net] On Behalf Of Johannes Schnitzler

Sent: Friday, June 28, 2013 10:54 PM

To: Knut Staring

Cc: dhis2-users@lists.launchpad.net; Ola Hodne Titlestad

Subject: Re: [Dhis2-users] Define week start day for Weekly periods

Dear All,

Is there anything which can be done in the near future to accommodate a week configuration (start day of the week and rule which week belongs to the new year)?

It would be good to have a generic approach:

Either allowing to define this in the set up of DHIS or to even allow it on a data set level which would open the possibility to use one DHIS installation by several countries.

If a generic approach is not feasible, would there be the possibility to define the dates and corresponding Year-Weeks in a table including dates/Year-Weeks in the future; allowing to select the correct week during data entry? How could this be handled in the analysis / how are week currently handled in the analysis?.

Otherwise this might be a showstopper for the use of DHIS.

Thank you!

Johannes

On 17 June 2013 09:57, Knut Staring knutst@gmail.com wrote:

Thanks Jim - Joda-Time certainly looks like a good candidate for non-ISO8601 calendar support, though I see this blog mention additional work needed for Nepal:

http://forjavaprogrammers.blogspot.com/2012/06/how-to-convert-nepali-date-to-english.html

Still, it may be possible to come up with a simpler workaround for Johannes’ problem which is limited to just weeks - perhaps most of the issues could there could be handled by relabeling the current ISO weeks?

Knut

On Mon, Jun 17, 2013 at 8:36 AM, Jim Grace jimgrace@gmail.com wrote:

I haven’t use it yet, but Joda-Time looks impressive at first glance. They claim it’s “the de facto standard library for advanced date and time in Java” and “The Gregorian, Julian, Buddhist, Coptic, Ethiopic and Islamic systems are also included” (besides the default ISO8601). http://joda.sourceforge.net/

Cheers,

Jim

On Jun 17, 2013, at 12:21 AM, Johannes Schnitzler schnitzlerj@gmail.com wrote:

Dear All,

Coming back to the possibility to define the different “type of weeks” in the setting.

Does anyone know a library in Java handling this easily?

In case this will not be addressed soon in the next versions. What would be the workaround for a given scenario/country - has anyone addressed this already?

Best regards,

Johannes

On 15 June 2013 18:18, Knut Staring knutst@gmail.com wrote:

I’m not sure it is important to aggregate along the time dimension for weekly data. But it is important to do so spatially, ie upwards in the hierarchy.

Sent from my mobile

On Jun 15, 2013 11:36 AM, “Bob Jolliffe” bobjolliffe@gmail.com wrote:

Aggregating from weeks to months will always be approximate so these anomalies will happen. Maybe some clever pro-rata scaling can be done.

On 15 June 2013 10:39, Ola Hodne Titlestad olati@ifi.uio.no wrote:

Not very related, but one issue with weeks that came up in Uganda recently is that monthly aggregates of the reporting rates of weekly datasets got above 100% for months with 5 weeks. Seems the aggregation code here assumes 4 weeks per month throughout the year and with 5 weeks of reported datasets for some months the reporting rate got too high.

Ola

On 15 Jun 2013 11:24, “John Lewis” johnlewis.hisp@gmail.com wrote:

Hi all,

Defining week start and end date is quite complicated. I had the same issue in bangladesh. The point also how do we aggregate it to month. Technically we break the data of a week which fall in two month, which is not a good idea.

Instead of rewriting.entire aggregation engine would it better to define as daily data set. Since we collect data once a week we can use daily data set and modify data enter screen to show weekly end date.

On 15 Jun 2013 03:40, “Johannes Schnitzler” schnitzlerj@gmail.com wrote:

Dear All,

How I have solved the issue in another application, also used for Cambodia, is that I’m allowing to define the start day of the week e.g. Monday, Tuesday, Wednesday … as well as how many days “touching the new year” are needed to qualify a week to be counted as week 1 of the new year. This is part of the setting of the application and is used to calculate it on the fly e.g. for data entry date / week selection. I have written the code for this in R (bad code… but it is working). This allows to deal with a lot of countries.

For ISO it is 4 days to qualify for the new year and the start day depends on the country,

For Cambodia it is 1 day and start day is Wednesday.

As in DHIS I’m storing the period type daily weekly monthly …

And for the actual date I’m storing always start date, end date of the period and in addition the Year and the TIme unit…

For weekly data: start date, end date, Week Number, Year of the week (which might be of course be different from the Year of the first day of the week for the first and last week of the year).

For daily data: start date, end date (which is of course the same), day number in year, Year

For monthly data: start date, end date, month number, Year

…

In this case one has all freedom to combine data sets from different “week definitions” as one can decide later on which week calculation to be applied across the piece based on the start date of the week knowing that the week counting might be off for some of the countries for a few days. But this is the best what we can do I assume …

Best regards,

Johannes

On 14 June 2013 19:08, Knut Staring knutst@gmail.com wrote:

Will have to investigate and get back to you

Sent from my mobile

On Jun 14, 2013 6:06 PM, “Bob Jolliffe” bobjolliffe@gmail.com wrote:

how do they determine which day is day 1 of week 1?

On 14 June 2013 17:52, Knut Staring knutst@gmail.com wrote:

For my purposes, that’s really all I know, and for my requirements it will be exactly parallel to countries that require Monday instead of Sunday.

But certainly the Nepali, Ethiopian and several others have more complex requirements, which I think partly must be treated as separate cases.

On Fri, Jun 14, 2013 at 5:42 PM, Bob Jolliffe bobjolliffe@gmail.com wrote:

Explain the Cambodian calendar a bit more. I am sure there is more to it than when the weeks start. Or is that it?

On 14 June 2013 17:35, Knut Staring knutst@gmail.com wrote:

Thanks Jason and Bob.

Unfortunately, this seems to be a pressing requirement in several countries

I think one simplification that could be justified would be to have only one week type in one DHIS2 instance. Thus, the exact meaning of 2012W25 is defined by looking at the FIRSTDAYOFWEEK setting in the particular db (which probably should be displayed on the About DHIS 2 page.

Of course, that does not solve interoperability issues (but rather it deteriorate through obfuscation), but I think it is reasonable to think that everyone in Cambodia uses weeks starting on Wednesdays.

On Fri, Jun 14, 2013 at 5:29 PM, Bob Jolliffe bobjolliffe@gmail.com wrote:

Weeks are as we all know something of a headache. What we have defined currently in DHIS2, as Jason points out, is the ISO standard week, which even itself is something of a headache.

I suspect that to incorporate all the variety which is called for (Wednesday week starts is the most exotic I’ve seen to date! But there are also Islamic calendar issues, Ethiopian/Amharic calendar, Monday vs Sunday starts etc) we might need to zoom right out to define a generic repeating period with rules around starting and stopping. But it could get quite complicated. Two immediate issues which come to mind

(i) we need a string representation of periods. So 2010W23 has a particular meaning in terms of the ISO week number. How to differentiate this from a week which starts on a Wednesday (and which day of the year would be day 1 of week 1?);

(ii) how to aggregate along the time axis ie, weeks->months->years

Its something which needs to be done, but I am not at all clear what the correct design approach would be. Would be interested to hear people’s thoughts.

Bob

On 14 June 2013 17:12, Knut Staring knutst@gmail.com wrote:

Devs: Is this something which could be targeted for 2.13?

Knut

On Mon, May 13, 2013 at 6:18 PM, Jason Pickering jason.p.pickering@gmail.com wrote:

Currently not. This has come up before, but for now, ISO weeks are what has been implemented.

http://en.wikipedia.org/wiki/ISO_week_date

Regards,

Jason

On Mon, May 13, 2013 at 6:54 PM, Paulo Grácio pgracio@criticalsoftware.com wrote:

Hi,

Is there a way to define week start day for Weekly periods?

Regards,

Paulo Grácio

Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp

–

Knut Staring

Dept. of Informatics, University of Oslo

+4791880522

http://dhis2.org

Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp

–

Knut Staring

Dept. of Informatics, University of Oslo

+4791880522

http://dhis2.org

–

Knut Staring

Dept. of Informatics, University of Oslo

+4791880522

http://dhis2.org

Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp

–

Knut Staring

Dept. of Informatics, University of Oslo

+4791880522

http://dhis2.org

jason · 30 June 2013 07:38

Hi Johannes,

If you can define the method which Cambodian weeks are generated, they could be implemented. Look at the other period classes, and you can see it is not that difficult, IF, there is some algorithm to actually generate them (which of course is the case for ISO weeks). Are you sure it is possible? In some calendric systems, intervals are not always so straight forward to define, and there are “experts” which decide what the months or other intervals may actually be.

I think having set of period classes for “User predefined periods” might solve a lot of these issues. Right now, the period classes and methods are used to algorithmically generate the start/end dates. What about if we just pre-created them and injected say 10 years work of them into the “period” table? Then DHIS2 would not have to worry about generating them, but would of course not be able to generate them for periods outside of the periods which already exist?

This might solve the issue we have with Nepali months as well, or other intervals which are for which no algorithm exists to derive future and past intervals, but which can be mapped onto the Gregorian calendar. Lars? Bob? Comments?

Regards,

Jason

···

On Sat, Jun 29, 2013 at 11:09 AM, Wilson,Randy rwilson@msh.org wrote:

Hi Johannes,

I’m not sure how DHIS-2 automatically creates the entries in the period table, but once they are created would one of the following options work?

· Pre-fill the period tables with weekly records for several years forward, then hack the period table to reset the start/end dates for weekly periods using an update query – adding or subtracting days depending upon when you want the start day.

· Create a period translation table (periodid, newstartdate, newenddate) that you can link to the datavalue table and produce your analyses according to the desired time frames.

In Rwanda we end up doing a lot of our analyses with custom SQL queries so option 2 would probably work for us – though users would have to be told to disregard the start date displayed during data entry – and focus on the week number.

Randy

From: Dhis2-users [mailto:dhis2-users-bounces+rwilson=msh.org@lists.launchpad.net] On Behalf Of Johannes Schnitzler

Sent: Friday, June 28, 2013 10:54 PM

To: Knut Staring

Cc: dhis2-users@lists.launchpad.net; Ola Hodne Titlestad

Subject: Re: [Dhis2-users] Define week start day for Weekly periods

Dear All,

Is there anything which can be done in the near future to accommodate a week configuration (start day of the week and rule which week belongs to the new year)?

It would be good to have a generic approach:

Either allowing to define this in the set up of DHIS or to even allow it on a data set level which would open the possibility to use one DHIS installation by several countries.

If a generic approach is not feasible, would there be the possibility to define the dates and corresponding Year-Weeks in a table including dates/Year-Weeks in the future; allowing to select the correct week during data entry? How could this be handled in the analysis / how are week currently handled in the analysis?.

Otherwise this might be a showstopper for the use of DHIS.

Thank you!

Johannes

On 17 June 2013 09:57, Knut Staring knutst@gmail.com wrote:

Thanks Jim - Joda-Time certainly looks like a good candidate for non-ISO8601 calendar support, though I see this blog mention additional work needed for Nepal:

http://forjavaprogrammers.blogspot.com/2012/06/how-to-convert-nepali-date-to-english.html

Still, it may be possible to come up with a simpler workaround for Johannes’ problem which is limited to just weeks - perhaps most of the issues could there could be handled by relabeling the current ISO weeks?

Knut

On Mon, Jun 17, 2013 at 8:36 AM, Jim Grace jimgrace@gmail.com wrote:

I haven’t use it yet, but Joda-Time looks impressive at first glance. They claim it’s “the de facto standard library for advanced date and time in Java” and “The Gregorian, Julian, Buddhist, Coptic, Ethiopic and Islamic systems are also included” (besides the default ISO8601). http://joda.sourceforge.net/

Cheers,

Jim

On Jun 17, 2013, at 12:21 AM, Johannes Schnitzler schnitzlerj@gmail.com wrote:

Dear All,

Coming back to the possibility to define the different “type of weeks” in the setting.

Does anyone know a library in Java handling this easily?

In case this will not be addressed soon in the next versions. What would be the workaround for a given scenario/country - has anyone addressed this already?

Best regards,

Johannes

On 15 June 2013 18:18, Knut Staring knutst@gmail.com wrote:

I’m not sure it is important to aggregate along the time dimension for weekly data. But it is important to do so spatially, ie upwards in the hierarchy.

Sent from my mobile

On Jun 15, 2013 11:36 AM, “Bob Jolliffe” bobjolliffe@gmail.com wrote:

Aggregating from weeks to months will always be approximate so these anomalies will happen. Maybe some clever pro-rata scaling can be done.

On 15 June 2013 10:39, Ola Hodne Titlestad olati@ifi.uio.no wrote:

Not very related, but one issue with weeks that came up in Uganda recently is that monthly aggregates of the reporting rates of weekly datasets got above 100% for months with 5 weeks. Seems the aggregation code here assumes 4 weeks per month throughout the year and with 5 weeks of reported datasets for some months the reporting rate got too high.

Ola

On 15 Jun 2013 11:24, “John Lewis” johnlewis.hisp@gmail.com wrote:

Hi all,

Defining week start and end date is quite complicated. I had the same issue in bangladesh. The point also how do we aggregate it to month. Technically we break the data of a week which fall in two month, which is not a good idea.

Instead of rewriting.entire aggregation engine would it better to define as daily data set. Since we collect data once a week we can use daily data set and modify data enter screen to show weekly end date.

On 15 Jun 2013 03:40, “Johannes Schnitzler” schnitzlerj@gmail.com wrote:

Dear All,

How I have solved the issue in another application, also used for Cambodia, is that I’m allowing to define the start day of the week e.g. Monday, Tuesday, Wednesday … as well as how many days “touching the new year” are needed to qualify a week to be counted as week 1 of the new year. This is part of the setting of the application and is used to calculate it on the fly e.g. for data entry date / week selection. I have written the code for this in R (bad code… but it is working). This allows to deal with a lot of countries.

For ISO it is 4 days to qualify for the new year and the start day depends on the country,

For Cambodia it is 1 day and start day is Wednesday.

As in DHIS I’m storing the period type daily weekly monthly …

And for the actual date I’m storing always start date, end date of the period and in addition the Year and the TIme unit…

For weekly data: start date, end date, Week Number, Year of the week (which might be of course be different from the Year of the first day of the week for the first and last week of the year).

For daily data: start date, end date (which is of course the same), day number in year, Year

For monthly data: start date, end date, month number, Year

…

In this case one has all freedom to combine data sets from different “week definitions” as one can decide later on which week calculation to be applied across the piece based on the start date of the week knowing that the week counting might be off for some of the countries for a few days. But this is the best what we can do I assume …

Best regards,

Johannes

On 14 June 2013 19:08, Knut Staring knutst@gmail.com wrote:

Will have to investigate and get back to you

Sent from my mobile

On Jun 14, 2013 6:06 PM, “Bob Jolliffe” bobjolliffe@gmail.com wrote:

how do they determine which day is day 1 of week 1?

On 14 June 2013 17:52, Knut Staring knutst@gmail.com wrote:

For my purposes, that’s really all I know, and for my requirements it will be exactly parallel to countries that require Monday instead of Sunday.

But certainly the Nepali, Ethiopian and several others have more complex requirements, which I think partly must be treated as separate cases.

On Fri, Jun 14, 2013 at 5:42 PM, Bob Jolliffe bobjolliffe@gmail.com wrote:

Explain the Cambodian calendar a bit more. I am sure there is more to it than when the weeks start. Or is that it?

On 14 June 2013 17:35, Knut Staring knutst@gmail.com wrote:

Thanks Jason and Bob.

Unfortunately, this seems to be a pressing requirement in several countries

I think one simplification that could be justified would be to have only one week type in one DHIS2 instance. Thus, the exact meaning of 2012W25 is defined by looking at the FIRSTDAYOFWEEK setting in the particular db (which probably should be displayed on the About DHIS 2 page.

Of course, that does not solve interoperability issues (but rather it deteriorate through obfuscation), but I think it is reasonable to think that everyone in Cambodia uses weeks starting on Wednesdays.

On Fri, Jun 14, 2013 at 5:29 PM, Bob Jolliffe bobjolliffe@gmail.com wrote:

Weeks are as we all know something of a headache. What we have defined currently in DHIS2, as Jason points out, is the ISO standard week, which even itself is something of a headache.

I suspect that to incorporate all the variety which is called for (Wednesday week starts is the most exotic I’ve seen to date! But there are also Islamic calendar issues, Ethiopian/Amharic calendar, Monday vs Sunday starts etc) we might need to zoom right out to define a generic repeating period with rules around starting and stopping. But it could get quite complicated. Two immediate issues which come to mind

(i) we need a string representation of periods. So 2010W23 has a particular meaning in terms of the ISO week number. How to differentiate this from a week which starts on a Wednesday (and which day of the year would be day 1 of week 1?);

(ii) how to aggregate along the time axis ie, weeks->months->years

Its something which needs to be done, but I am not at all clear what the correct design approach would be. Would be interested to hear people’s thoughts.

Bob

On 14 June 2013 17:12, Knut Staring knutst@gmail.com wrote:

Devs: Is this something which could be targeted for 2.13?

Knut

On Mon, May 13, 2013 at 6:18 PM, Jason Pickering jason.p.pickering@gmail.com wrote:

Currently not. This has come up before, but for now, ISO weeks are what has been implemented.

http://en.wikipedia.org/wiki/ISO_week_date

Regards,

Jason

On Mon, May 13, 2013 at 6:54 PM, Paulo Grácio pgracio@criticalsoftware.com wrote:

Hi,

Is there a way to define week start day for Weekly periods?

Regards,

Paulo Grácio

Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp

–

Knut Staring

Dept. of Informatics, University of Oslo

+4791880522

http://dhis2.org

Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp

–

Knut Staring

Dept. of Informatics, University of Oslo

+4791880522

http://dhis2.org

–

Knut Staring

Dept. of Informatics, University of Oslo

+4791880522

http://dhis2.org

Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp

–

Knut Staring

Dept. of Informatics, University of Oslo

+4791880522

http://dhis2.org

Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp

Johannes_Schnitzler · 1 July 2013 11:39

Dear Jason,

Thank you for the reply.

It can be calculated; Start on Wednesday, if at least one day of a given week is in the “new Year” the week is regarded to belong to the “new Year”.

In addition there have been libraries for Java mentioned in the past which could handle quite a bit of the necessary calculations.

Adding the possibility to define in addition “custom” periods would handle all.

The question remains how the analysis for weekly data would work then.

If i’m not mistaken at this point in time the period table stores just the from/to dates and the type of period e.g. 2 for weekly. The information about actual week number and the corresponding Year is not stored in the periods table.

Best regards,

Johannes

···

On 30 June 2013 09:38, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Johannes,

If you can define the method which Cambodian weeks are generated, they could be implemented. Look at the other period classes, and you can see it is not that difficult, IF, there is some algorithm to actually generate them (which of course is the case for ISO weeks). Are you sure it is possible? In some calendric systems, intervals are not always so straight forward to define, and there are “experts” which decide what the months or other intervals may actually be.

I think having set of period classes for “User predefined periods” might solve a lot of these issues. Right now, the period classes and methods are used to algorithmically generate the start/end dates. What about if we just pre-created them and injected say 10 years work of them into the “period” table? Then DHIS2 would not have to worry about generating them, but would of course not be able to generate them for periods outside of the periods which already exist?

This might solve the issue we have with Nepali months as well, or other intervals which are for which no algorithm exists to derive future and past intervals, but which can be mapped onto the Gregorian calendar. Lars? Bob? Comments?

Regards,

Jason

On Sat, Jun 29, 2013 at 11:09 AM, Wilson,Randy rwilson@msh.org wrote:

Hi Johannes,

I’m not sure how DHIS-2 automatically creates the entries in the period table, but once they are created would one of the following options work?

· Pre-fill the period tables with weekly records for several years forward, then hack the period table to reset the start/end dates for weekly periods using an update query – adding or subtracting days depending upon when you want the start day.

· Create a period translation table (periodid, newstartdate, newenddate) that you can link to the datavalue table and produce your analyses according to the desired time frames.

In Rwanda we end up doing a lot of our analyses with custom SQL queries so option 2 would probably work for us – though users would have to be told to disregard the start date displayed during data entry – and focus on the week number.

Randy

From: Dhis2-users [mailto:dhis2-users-bounces+rwilson=msh.org@lists.launchpad.net] On Behalf Of Johannes Schnitzler

Sent: Friday, June 28, 2013 10:54 PM

To: Knut Staring

Cc: dhis2-users@lists.launchpad.net; Ola Hodne Titlestad

Subject: Re: [Dhis2-users] Define week start day for Weekly periods

Dear All,

Is there anything which can be done in the near future to accommodate a week configuration (start day of the week and rule which week belongs to the new year)?

It would be good to have a generic approach:

Either allowing to define this in the set up of DHIS or to even allow it on a data set level which would open the possibility to use one DHIS installation by several countries.

If a generic approach is not feasible, would there be the possibility to define the dates and corresponding Year-Weeks in a table including dates/Year-Weeks in the future; allowing to select the correct week during data entry? How could this be handled in the analysis / how are week currently handled in the analysis?.

Otherwise this might be a showstopper for the use of DHIS.

Thank you!

Johannes

On 17 June 2013 09:57, Knut Staring knutst@gmail.com wrote:

Thanks Jim - Joda-Time certainly looks like a good candidate for non-ISO8601 calendar support, though I see this blog mention additional work needed for Nepal:

http://forjavaprogrammers.blogspot.com/2012/06/how-to-convert-nepali-date-to-english.html

Still, it may be possible to come up with a simpler workaround for Johannes’ problem which is limited to just weeks - perhaps most of the issues could there could be handled by relabeling the current ISO weeks?

Knut

On Mon, Jun 17, 2013 at 8:36 AM, Jim Grace jimgrace@gmail.com wrote:

I haven’t use it yet, but Joda-Time looks impressive at first glance. They claim it’s “the de facto standard library for advanced date and time in Java” and “The Gregorian, Julian, Buddhist, Coptic, Ethiopic and Islamic systems are also included” (besides the default ISO8601). http://joda.sourceforge.net/

Cheers,

Jim

On Jun 17, 2013, at 12:21 AM, Johannes Schnitzler schnitzlerj@gmail.com wrote:

Dear All,

Coming back to the possibility to define the different “type of weeks” in the setting.

Does anyone know a library in Java handling this easily?

In case this will not be addressed soon in the next versions. What would be the workaround for a given scenario/country - has anyone addressed this already?

Best regards,

Johannes

On 15 June 2013 18:18, Knut Staring knutst@gmail.com wrote:

I’m not sure it is important to aggregate along the time dimension for weekly data. But it is important to do so spatially, ie upwards in the hierarchy.

Sent from my mobile

On Jun 15, 2013 11:36 AM, “Bob Jolliffe” bobjolliffe@gmail.com wrote:

Aggregating from weeks to months will always be approximate so these anomalies will happen. Maybe some clever pro-rata scaling can be done.

On 15 June 2013 10:39, Ola Hodne Titlestad olati@ifi.uio.no wrote:

Not very related, but one issue with weeks that came up in Uganda recently is that monthly aggregates of the reporting rates of weekly datasets got above 100% for months with 5 weeks. Seems the aggregation code here assumes 4 weeks per month throughout the year and with 5 weeks of reported datasets for some months the reporting rate got too high.

Ola

On 15 Jun 2013 11:24, “John Lewis” johnlewis.hisp@gmail.com wrote:

Hi all,

Defining week start and end date is quite complicated. I had the same issue in bangladesh. The point also how do we aggregate it to month. Technically we break the data of a week which fall in two month, which is not a good idea.

Instead of rewriting.entire aggregation engine would it better to define as daily data set. Since we collect data once a week we can use daily data set and modify data enter screen to show weekly end date.

On 15 Jun 2013 03:40, “Johannes Schnitzler” schnitzlerj@gmail.com wrote:

Dear All,

How I have solved the issue in another application, also used for Cambodia, is that I’m allowing to define the start day of the week e.g. Monday, Tuesday, Wednesday … as well as how many days “touching the new year” are needed to qualify a week to be counted as week 1 of the new year. This is part of the setting of the application and is used to calculate it on the fly e.g. for data entry date / week selection. I have written the code for this in R (bad code… but it is working). This allows to deal with a lot of countries.

For ISO it is 4 days to qualify for the new year and the start day depends on the country,

For Cambodia it is 1 day and start day is Wednesday.

As in DHIS I’m storing the period type daily weekly monthly …

And for the actual date I’m storing always start date, end date of the period and in addition the Year and the TIme unit…

For weekly data: start date, end date, Week Number, Year of the week (which might be of course be different from the Year of the first day of the week for the first and last week of the year).

For daily data: start date, end date (which is of course the same), day number in year, Year

For monthly data: start date, end date, month number, Year

…

In this case one has all freedom to combine data sets from different “week definitions” as one can decide later on which week calculation to be applied across the piece based on the start date of the week knowing that the week counting might be off for some of the countries for a few days. But this is the best what we can do I assume …

Best regards,

Johannes

On 14 June 2013 19:08, Knut Staring knutst@gmail.com wrote:

Will have to investigate and get back to you

Sent from my mobile

On Jun 14, 2013 6:06 PM, “Bob Jolliffe” bobjolliffe@gmail.com wrote:

how do they determine which day is day 1 of week 1?

On 14 June 2013 17:52, Knut Staring knutst@gmail.com wrote:

For my purposes, that’s really all I know, and for my requirements it will be exactly parallel to countries that require Monday instead of Sunday.

But certainly the Nepali, Ethiopian and several others have more complex requirements, which I think partly must be treated as separate cases.

On Fri, Jun 14, 2013 at 5:42 PM, Bob Jolliffe bobjolliffe@gmail.com wrote:

Explain the Cambodian calendar a bit more. I am sure there is more to it than when the weeks start. Or is that it?

On 14 June 2013 17:35, Knut Staring knutst@gmail.com wrote:

Thanks Jason and Bob.

Unfortunately, this seems to be a pressing requirement in several countries

I think one simplification that could be justified would be to have only one week type in one DHIS2 instance. Thus, the exact meaning of 2012W25 is defined by looking at the FIRSTDAYOFWEEK setting in the particular db (which probably should be displayed on the About DHIS 2 page.

Of course, that does not solve interoperability issues (but rather it deteriorate through obfuscation), but I think it is reasonable to think that everyone in Cambodia uses weeks starting on Wednesdays.

On Fri, Jun 14, 2013 at 5:29 PM, Bob Jolliffe bobjolliffe@gmail.com wrote:

Weeks are as we all know something of a headache. What we have defined currently in DHIS2, as Jason points out, is the ISO standard week, which even itself is something of a headache.

I suspect that to incorporate all the variety which is called for (Wednesday week starts is the most exotic I’ve seen to date! But there are also Islamic calendar issues, Ethiopian/Amharic calendar, Monday vs Sunday starts etc) we might need to zoom right out to define a generic repeating period with rules around starting and stopping. But it could get quite complicated. Two immediate issues which come to mind

(i) we need a string representation of periods. So 2010W23 has a particular meaning in terms of the ISO week number. How to differentiate this from a week which starts on a Wednesday (and which day of the year would be day 1 of week 1?);

(ii) how to aggregate along the time axis ie, weeks->months->years

Its something which needs to be done, but I am not at all clear what the correct design approach would be. Would be interested to hear people’s thoughts.

Bob

On 14 June 2013 17:12, Knut Staring knutst@gmail.com wrote:

Devs: Is this something which could be targeted for 2.13?

Knut

On Mon, May 13, 2013 at 6:18 PM, Jason Pickering jason.p.pickering@gmail.com wrote:

Currently not. This has come up before, but for now, ISO weeks are what has been implemented.

http://en.wikipedia.org/wiki/ISO_week_date

Regards,

Jason

On Mon, May 13, 2013 at 6:54 PM, Paulo Grácio pgracio@criticalsoftware.com wrote:

Hi,

Is there a way to define week start day for Weekly periods?

Regards,

Paulo Grácio

Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp

–

Knut Staring

Dept. of Informatics, University of Oslo

+4791880522

http://dhis2.org

Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp

–

Knut Staring

Dept. of Informatics, University of Oslo

+4791880522

http://dhis2.org

–

Knut Staring

Dept. of Informatics, University of Oslo

+4791880522

http://dhis2.org

Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp

–

Knut Staring

Dept. of Informatics, University of Oslo

+4791880522

http://dhis2.org

Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp

Lars · 2 July 2013 12:09

Hello Johannes,

the way we handle time period aggregation is simple: When aggregating in the time dimension, we include all periods which start dates fall within the aggregation period. As an example, when aggregating monthly data to a quarterly aggregate, we include all months which start dates fall within that quarter. This usually works well but is admittedly not ideal for weeks, where you might want more specialized rules for determining whether a week falls within a “longer” period or not.

At the moment, the design of our period solution allows only for fixed period types, in the sense that you can select from a predefined lists of period types; you cannot select “weekly” and then start day of week.

It seems you have two requirements. i) Displaying correct Cambodian weekly start/end dates in data entry and ii) have the time period aggregation work correctly in terms of which weeks get included in aggregation periods. Correct?

I think we have two options here. The first is to allow more flexibility in terms of period starting days and which day of week determines which year a week falls within. This would require a bit development work. The second is to implement more predefined period types, ie. “Weekly (Wednesday)”. The issue with this is that we might end up with 7 different weekly period types. Anyone knows how many types of weeks there are out there?

regards,

Lars

Johan_Ivar_Saebo · 2 July 2013 13:37

Hi Lars,

there are many different types of weeks, but most of them can be safely ignored. But since we tried to sort this out for CAREC earlier, let me take the opportunity again to bring up these common weeks:

So I would suggest a more flexible way of determining your weeks, since there are issues of a) which day the week starts on, b) which year the week should belong to, c) if the weeknumber is assigned twice (two week 1). Maybe it does not have to be a generic solution, but at least the most common calendars should be supported (but then we might be surprised over have many there might be)

Regards from Dhaka (week start on sunday)
Johan

···

On 02.07.2013 14:09, Lars Helge �verland wrote:

Hello Johannes,

the way we handle time period aggregation is simple: When aggregating in
the time dimension, we include all periods which start dates fall within
the aggregation period. As an example, when aggregating monthly data to
a quarterly aggregate, we include all months which start dates fall
within that quarter. This usually works well but is admittedly not ideal
for weeks, where you might want more specialized rules for determining
whether a week falls within a "longer" period or not.

At the moment, the design of our period solution allows only for fixed
period types, in the sense that you can select from a predefined lists
of period types; you cannot select "weekly" and then start day of week.

It seems you have two requirements. i) Displaying correct Cambodian
weekly start/end dates in data entry and ii) have the time period
aggregation work correctly in terms of which weeks get included in
aggregation periods. Correct?

I think we have two options here. The first is to allow more flexibility
in terms of period starting days and which day of week determines which
year a week falls within. This would require a bit development work. The
second is to implement more predefined period types, ie. "Weekly
(Wednesday)". The issue with this is that we might end up with 7
different weekly period types. Anyone knows how many types of weeks
there are out there?

regards,

Lars

_______________________________________________
Mailing list: DHIS 2 Users in Launchpad
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : DHIS 2 Users in Launchpad
More help : ListHelp - Launchpad Help

jason · 2 July 2013 13:57

Thanks for the explanation Lars. Could you comment on the possibility to include user defined periods which would need to be generated externally and then imported? Generating a series of weeks which start on Wednesday would be a few lines of code in R. I think this might keep things simple,but allow implementers with specific requiememnts to manually generate periods for as far into the past or future as they may need.

Of course the issue of period names is a bit complicated,but likely easily solvable?

Regards,

Jason

···

–

Sent from my mobile

On Jul 2, 2013 2:09 PM, “Lars Helge Øverland” larshelge@gmail.com wrote:

Hello Johannes,

the way we handle time period aggregation is simple: When aggregating in the time dimension, we include all periods which start dates fall within the aggregation period. As an example, when aggregating monthly data to a quarterly aggregate, we include all months which start dates fall within that quarter. This usually works well but is admittedly not ideal for weeks, where you might want more specialized rules for determining whether a week falls within a “longer” period or not.

At the moment, the design of our period solution allows only for fixed period types, in the sense that you can select from a predefined lists of period types; you cannot select “weekly” and then start day of week.

It seems you have two requirements. i) Displaying correct Cambodian weekly start/end dates in data entry and ii) have the time period aggregation work correctly in terms of which weeks get included in aggregation periods. Correct?

I think we have two options here. The first is to allow more flexibility in terms of period starting days and which day of week determines which year a week falls within. This would require a bit development work. The second is to implement more predefined period types, ie. “Weekly (Wednesday)”. The issue with this is that we might end up with 7 different weekly period types. Anyone knows how many types of weeks there are out there?

regards,

Lars

Johannes_Schnitzler · 2 July 2013 13:59

Dear Lars,

I don’t know how many week types are out there but I assume the most common combinations are diffrent start days of the week + the number of days “needed” in the “new year” to qualify this week to the “new Year”. The most common will be the same rule like for ISO weeks, 4 days in the new year will determine that this week belongs to the new year, OR 1 day touching the new year is sufficient to qualify for the new year.

For weekly data you would of course like to see (e.g. on a charts or tables) Week 53 Year 2010, Week 1 2011 …

For this to happen the information needs to be stored in DHIS / or calculated on the fly which week is a specific date (e.g. start date of the week).

It would be best if for weekly collected data there could eb the option to dislpaly by “date” or by Week-Year (Week 53 Year 2010, Week 01 Year 2011 …)

For the aggregation the data could also be aggregated - e.g. for the Year - there would be the possibility to choose aggregation by WEEK-YEAR or the actual dates.

The WEEK-YEAR aggregation would make sense to show the total for the different Years.

The “date” manipulation would be needed for e.g. time series analysis …

The configuration of the start and end date and the week + Year is of course possible with a static table(s) which can be filled for several years in the past and in the future.

Nevertheless the question of course reamins how this can be handled in the anlysis / aggregation.

I hope ther are some ideas how this can be implemented as I don’t want to that this is the reason to take DHIS.

Thank you,

Joahnnes

Johannes

···

On 2 July 2013 14:09, Lars Helge Øverland larshelge@gmail.com wrote:

Hello Johannes,

the way we handle time period aggregation is simple: When aggregating in the time dimension, we include all periods which start dates fall within the aggregation period. As an example, when aggregating monthly data to a quarterly aggregate, we include all months which start dates fall within that quarter. This usually works well but is admittedly not ideal for weeks, where you might want more specialized rules for determining whether a week falls within a “longer” period or not.

At the moment, the design of our period solution allows only for fixed period types, in the sense that you can select from a predefined lists of period types; you cannot select “weekly” and then start day of week.

It seems you have two requirements. i) Displaying correct Cambodian weekly start/end dates in data entry and ii) have the time period aggregation work correctly in terms of which weeks get included in aggregation periods. Correct?

I think we have two options here. The first is to allow more flexibility in terms of period starting days and which day of week determines which year a week falls within. This would require a bit development work. The second is to implement more predefined period types, ie. “Weekly (Wednesday)”. The issue with this is that we might end up with 7 different weekly period types. Anyone knows how many types of weeks there are out there?

regards,

Lars