This may have been brought up before, but we get some data sometimes, submitted in faraway time periods (like 1-1-0001). No clue really how this gets in there, but it exposes a weakness in the analytics, in that if data exists from these ancient (or far in the future time periods), many thousands of tables will be created (since a table will be created for each year, based on the minimum and maximum period).
I would like to suggest that we implement a mechanism (perhaps a system setting) to restrict the analytics to certain time ranges, namely
I see the problem. Analytics after 2.14 should remove all empty analytics tables though so you should not have all those empty tables between 0 and now.
Unless someone plans to use DHIS for archaeological research / studies of the ancient Greeks we could maybe hard code some sensible limits to protect from those extreme cases.
This may have been brought up before, but we get some data sometimes, submitted in faraway time periods (like 1-1-0001). No clue really how this gets in there, but it exposes a weakness in the analytics, in that if data exists from these ancient (or far in the future time periods), many thousands of tables will be created (since a table will be created for each year, based on the minimum and maximum period).
I would like to suggest that we implement a mechanism (perhaps a system setting) to restrict the analytics to certain time ranges, namely
Yeah, this is really sort of an issue with the submission of mobile data from phones who have not set their clocks correctly, and the fact that DHIS2 does not really barf at recording things for periods from the ancient past. Although we try to instruct people this is necessary, just one submission at Jan 1, 0000, will increase the time which it takes to do the analytics. I think some reasonable, as you say, limits would be good. Those who want to use DHIS2 for archeaological purposes, could always change these hard-coded limits.
I would suggest 1900-2100, which should be more than generous enough for the vast majority of use cases. We are looking really just to restrict these extreme cases.
Regards,
Jason
···
On Thu, Jun 12, 2014 at 2:40 PM, Lars Helge Øverland larshelge@gmail.com wrote:
Hi Jason,
I see the problem. Analytics after 2.14 should remove all empty analytics tables though so you should not have all those empty tables between 0 and now.
Unless someone plans to use DHIS for archaeological research / studies of the ancient Greeks we could maybe hard code some sensible limits to protect from those extreme cases.
This may have been brought up before, but we get some data sometimes, submitted in faraway time periods (like 1-1-0001). No clue really how this gets in there, but it exposes a weakness in the analytics, in that if data exists from these ancient (or far in the future time periods), many thousands of tables will be created (since a table will be created for each year, based on the minimum and maximum period).
I would like to suggest that we implement a mechanism (perhaps a system setting) to restrict the analytics to certain time ranges, namely
Yeah, this is really sort of an issue with the submission of mobile data from phones who have not set their clocks correctly, and the fact that DHIS2 does not really barf at recording things for periods from the ancient past. Although we try to instruct people this is necessary, just one submission at Jan 1, 0000, will increase the time which it takes to do the analytics. I think some reasonable, as you say, limits would be good. Those who want to use DHIS2 for archeaological purposes, could always change these hard-coded limits.
I would suggest 1900-2100, which should be more than generous enough for the vast majority of use cases. We are looking really just to restrict these extreme cases.
Regards,
Jason
On Thu, Jun 12, 2014 at 2:40 PM, Lars Helge Øverland larshelge@gmail.com wrote:
Hi Jason,
I see the problem. Analytics after 2.14 should remove all empty analytics tables though so you should not have all those empty tables between 0 and now.
Unless someone plans to use DHIS for archaeological research / studies of the ancient Greeks we could maybe hard code some sensible limits to protect from those extreme cases.
This may have been brought up before, but we get some data sometimes, submitted in faraway time periods (like 1-1-0001). No clue really how this gets in there, but it exposes a weakness in the analytics, in that if data exists from these ancient (or far in the future time periods), many thousands of tables will be created (since a table will be created for each year, based on the minimum and maximum period).
I would like to suggest that we implement a mechanism (perhaps a system setting) to restrict the analytics to certain time ranges, namely