Not sure if this a bug or feature, but each time I access the
Dashboard in 2.0.5, charts that have been added to the dashboard seem
to be regenerated. The charts involve a fair a bit of aggregation, so
they take several seconds (minutes actually in one case) to generate.
Is this the intended behaviour?
I would think it would make more sense to use a pre-generated copy for
the dashboard?
Regards,
Jason
--
Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+260968395190
Agree. We should rather add one or more “Refresh” buttons.
Knut (from my phone)
On Nov 29, 2010 6:17 AM, “Jason Pickering” jason.p.pickering@gmail.com wrote:
Not sure if this a bug or feature, but each time I access the
Dashboard in 2.0.5, charts that have been added to the dashboard seem
to be regenerated. The charts involve a fair a bit of aggregation, so
they take several seconds (minutes actually in one case) to generate.
Is this the intended behaviour?
I would think it would make more sense to use a pre-generated copy for
the dashboard?
Regards,
Jason
–
Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+260968395190
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help : https://help.launchpad.net/ListHelp
Created a blueprint here..
https://blueprints.launchpad.net/dhis2/+spec/cached-dashboard-charts
On Mon, Nov 29, 2010 at 9:03 AM, Knut Staring <knutst@gmail.com> wrote:
Agree. We should rather add one or more "Refresh" buttons.
Knut (from my phone)
On Nov 29, 2010 6:17 AM, "Jason Pickering" <jason.p.pickering@gmail.com> > wrote:
Not sure if this a bug or feature, but each time I access the
Dashboard in 2.0.5, charts that have been added to the dashboard seem
to be regenerated. The charts involve a fair a bit of aggregation, so
they take several seconds (minutes actually in one case) to generate.Is this the intended behaviour?
I would think it would make more sense to use a pre-generated copy for
the dashboard?Regards,
Jason--
Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+260968395190_______________________________________________
Mailing list: DHIS 2 developers in Launchpad
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : DHIS 2 developers in Launchpad
More help : ListHelp - Launchpad Help
--
Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+260968395190
Hi,
I’ve discussed this with Lars a couple of times and there doesn’t seem to be an easy fix here. My fix for now is to use charts that require less aggregation (e.g. reduce either period or orgunit dimension). Things like national yearly charts are more or less useless in the dashboard at the moment, but since it is a dashboard the latest month might be good enough (you can use relative periods to get this).
Lars can add more detail, but what I understood is that since the charts are generated off on-the-fly data sources using the aggregation service and not pre-generated values in datamart there is no easy way to persist them (basically that would be a data mart).
Having charts depend on data mart would make them faster to display on the dashboard, but then you might run the risk of data not being available in the datamart, which is why we moved many of the report tools from datamart to aggregation service some time back. With a more automated and controlled data mart (coming soon) this becomes a much more attractive option again.
This is part of a more general discussion on on-the-fly data processing (aggregation service) vs. pre-processed data (data mart) for all our reporting/presentation tools, and it relates to the type of system that is deployed, e.g. online servers with many users vs. offline install. We have talked about making it optional whether to use on-the-fly or datamart for all report tools as part of a system setting. Going for datamart makes a lot of sense in multiuser settings with lots of data requests and heavy data processing needed. We are moving towards a better managed/controlled data mart where we have scheduled exports that can make sure it is always up to date (e.g. nightly builds), but still that does not mean instant as in going directly from data entry to a report/chart.
Ola,
Ola Hodne Titlestad (Mr)
HISP
Department of Informatics
University of Oslo
Mobile: +47 48069736
Home address: Vetlandsvn. 95B, 0685 Oslo, Norway. Googlemaps link
On 29 November 2010 09:29, Jason Pickering jason.p.pickering@gmail.com wrote:
Created a blueprint here…
https://blueprints.launchpad.net/dhis2/+spec/cached-dashboard-charts
On Mon, Nov 29, 2010 at 9:03 AM, Knut Staring knutst@gmail.com wrote:
Agree. We should rather add one or more “Refresh” buttons.
Knut (from my phone)
On Nov 29, 2010 6:17 AM, “Jason Pickering” jason.p.pickering@gmail.com > > > wrote:
Not sure if this a bug or feature, but each time I access the
Dashboard in 2.0.5, charts that have been added to the dashboard seem
to be regenerated. The charts involve a fair a bit of aggregation, so
they take several seconds (minutes actually in one case) to generate.
Is this the intended behaviour?
I would think it would make more sense to use a pre-generated copy for
the dashboard?
Regards,
Jason
–
Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+260968395190
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help : https://help.launchpad.net/ListHelp
–
Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+260968395190
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help : https://help.launchpad.net/ListHelp
Agree that we need some deep thinking about datamarts and persistence
for 2.0.7, especially as we consider supporting dissemination use
cases - i.e. public data repositories.
For the case of charts, I suppose an alternative could be to store the
actual images (though there may be very many of those - and we
probably need to delete the ones no longer used in dashboards). This
should be more responsive than generating from datamart.
Knut
On Mon, Nov 29, 2010 at 9:58 AM, Ola Hodne Titlestad <olati@ifi.uio.no> wrote:
Hi,
I've discussed this with Lars a couple of times and there doesn't seem to be
an easy fix here. My fix for now is to use charts that require less
aggregation (e.g. reduce either period or orgunit dimension). Things like
national yearly charts are more or less useless in the dashboard at the
moment, but since it is a dashboard the latest month might be good enough
(you can use relative periods to get this).
Lars can add more detail, but what I understood is that since the charts are
generated off on-the-fly data sources using the aggregation service and not
pre-generated values in datamart there is no easy way to persist them
(basically that would be a data mart).
Having charts depend on data mart would make them faster to display on the
dashboard, but then you might run the risk of data not being available in
the datamart, which is why we moved many of the report tools from datamart
to aggregation service some time back. With a more automated and controlled
data mart (coming soon) this becomes a much more attractive option again.
This is part of a more general discussion on on-the-fly data processing
(aggregation service) vs. pre-processed data (data mart) for all our
reporting/presentation tools, and it relates to the type of system that is
deployed, e.g. online servers with many users vs. offline install. We have
talked about making it optional whether to use on-the-fly or datamart for
all report tools as part of a system setting. Going for datamart makes a lot
of sense in multiuser settings with lots of data requests and heavy data
processing needed. We are moving towards a better managed/controlled data
mart where we have scheduled exports that can make sure it is always up to
date (e.g. nightly builds), but still that does not mean instant as in going
directly from data entry to a report/chart.
Ola,----------------------------------
Ola Hodne Titlestad (Mr)
HISP
Department of Informatics
University of OsloMobile: +47 48069736
Home address: Vetlandsvn. 95B, 0685 Oslo, Norway. Googlemaps linkOn 29 November 2010 09:29, Jason Pickering <jason.p.pickering@gmail.com> > wrote:
Created a blueprint here..
https://blueprints.launchpad.net/dhis2/+spec/cached-dashboard-charts
On Mon, Nov 29, 2010 at 9:03 AM, Knut Staring <knutst@gmail.com> wrote:
> Agree. We should rather add one or more "Refresh" buttons.
>
> Knut (from my phone)
>
> On Nov 29, 2010 6:17 AM, "Jason Pickering" <jason.p.pickering@gmail.com> >> > wrote:
>
> Not sure if this a bug or feature, but each time I access the
> Dashboard in 2.0.5, charts that have been added to the dashboard seem
> to be regenerated. The charts involve a fair a bit of aggregation, so
> they take several seconds (minutes actually in one case) to generate.
>
> Is this the intended behaviour?
>
> I would think it would make more sense to use a pre-generated copy for
> the dashboard?
>
>
> Regards,
> Jason
>
>
> --
> Jason P. Pickering
> email: jason.p.pickering@gmail.com
> tel:+260968395190
>
> _______________________________________________
> Mailing list: DHIS 2 developers in Launchpad
> Post to : dhis2-devs@lists.launchpad.net
> Unsubscribe : DHIS 2 developers in Launchpad
> More help : ListHelp - Launchpad Help
>--
Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+260968395190_______________________________________________
Mailing list: DHIS 2 developers in Launchpad
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : DHIS 2 developers in Launchpad
More help : ListHelp - Launchpad Help
--
Cheers,
Knut Staring
Agree that we need some deep thinking about datamarts and persistence
for 2.0.7, especially as we consider supporting dissemination use
cases - i.e. public data repositories.For the case of charts, I suppose an alternative could be to store the
actual images (though there may be very many of those - and we
probably need to delete the ones no longer used in dashboards). This
should be more responsive than generating from datamart.
On the topic of Dashboards, I have been thinking about the inclusion
of thematic maps, and again there are similar considerations - one
could in fact store and display the images themselves, or we could
store geojson which includes coloring information in the mapview
table, making it similar to a datamart for maps, which would allow for
slightly interactive maps in the dashboard (e.g. mouse over a
suborgunit and see its name and value).
Knut
On Mon, Nov 29, 2010 at 10:04 AM, Knut Staring <knutst@gmail.com> wrote:
Knut
On Mon, Nov 29, 2010 at 9:58 AM, Ola Hodne Titlestad <olati@ifi.uio.no> wrote:
Hi,
I've discussed this with Lars a couple of times and there doesn't seem to be
an easy fix here. My fix for now is to use charts that require less
aggregation (e.g. reduce either period or orgunit dimension). Things like
national yearly charts are more or less useless in the dashboard at the
moment, but since it is a dashboard the latest month might be good enough
(you can use relative periods to get this).
Lars can add more detail, but what I understood is that since the charts are
generated off on-the-fly data sources using the aggregation service and not
pre-generated values in datamart there is no easy way to persist them
(basically that would be a data mart).
Having charts depend on data mart would make them faster to display on the
dashboard, but then you might run the risk of data not being available in
the datamart, which is why we moved many of the report tools from datamart
to aggregation service some time back. With a more automated and controlled
data mart (coming soon) this becomes a much more attractive option again.
This is part of a more general discussion on on-the-fly data processing
(aggregation service) vs. pre-processed data (data mart) for all our
reporting/presentation tools, and it relates to the type of system that is
deployed, e.g. online servers with many users vs. offline install. We have
talked about making it optional whether to use on-the-fly or datamart for
all report tools as part of a system setting. Going for datamart makes a lot
of sense in multiuser settings with lots of data requests and heavy data
processing needed. We are moving towards a better managed/controlled data
mart where we have scheduled exports that can make sure it is always up to
date (e.g. nightly builds), but still that does not mean instant as in going
directly from data entry to a report/chart.
Ola,----------------------------------
Ola Hodne Titlestad (Mr)
HISP
Department of Informatics
University of OsloMobile: +47 48069736
Home address: Vetlandsvn. 95B, 0685 Oslo, Norway. Googlemaps linkOn 29 November 2010 09:29, Jason Pickering <jason.p.pickering@gmail.com> >> wrote:
Created a blueprint here..
https://blueprints.launchpad.net/dhis2/+spec/cached-dashboard-charts
On Mon, Nov 29, 2010 at 9:03 AM, Knut Staring <knutst@gmail.com> wrote:
> Agree. We should rather add one or more "Refresh" buttons.
>
> Knut (from my phone)
>
> On Nov 29, 2010 6:17 AM, "Jason Pickering" <jason.p.pickering@gmail.com> >>> > wrote:
>
> Not sure if this a bug or feature, but each time I access the
> Dashboard in 2.0.5, charts that have been added to the dashboard seem
> to be regenerated. The charts involve a fair a bit of aggregation, so
> they take several seconds (minutes actually in one case) to generate.
>
> Is this the intended behaviour?
>
> I would think it would make more sense to use a pre-generated copy for
> the dashboard?
>
>
> Regards,
> Jason
>
>
> --
> Jason P. Pickering
> email: jason.p.pickering@gmail.com
> tel:+260968395190
>
> _______________________________________________
> Mailing list: DHIS 2 developers in Launchpad
> Post to : dhis2-devs@lists.launchpad.net
> Unsubscribe : DHIS 2 developers in Launchpad
> More help : ListHelp - Launchpad Help
>--
Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+260968395190_______________________________________________
Mailing list: DHIS 2 developers in Launchpad
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : DHIS 2 developers in Launchpad
More help : ListHelp - Launchpad Help--
Cheers,
Knut Staring
--
Cheers,
Knut Staring
Hi there.
Yeah, I can see how this is a bit tricky. The chart really has no idea
if its data is present or not in the datamart.
I guess I was really getting at something slightly different however,
which was simply the ability to have the option to cache the image,
and regenerate it only on request, similar to the report tables.
Whether this regeneration occurs through the data mart or an
on-the-fly aggregation procedure is really a different issue. I am
sure in a multiuser environment, this operation is going to bring the
system to a halt very quickly, if the wrong types of charts are
chosen. Since users have the ability to create their own charts as
well, they may not be very judicious about what types they create.
Caching the image on disk or as a BLOB in the DB, would seem not to be
too difficult?
Regards,
Jason
On Mon, Nov 29, 2010 at 10:58 AM, Ola Hodne Titlestad <olati@ifi.uio.no> wrote:
Hi,
I've discussed this with Lars a couple of times and there doesn't seem to be
an easy fix here. My fix for now is to use charts that require less
aggregation (e.g. reduce either period or orgunit dimension). Things like
national yearly charts are more or less useless in the dashboard at the
moment, but since it is a dashboard the latest month might be good enough
(you can use relative periods to get this).
Lars can add more detail, but what I understood is that since the charts are
generated off on-the-fly data sources using the aggregation service and not
pre-generated values in datamart there is no easy way to persist them
(basically that would be a data mart).
Having charts depend on data mart would make them faster to display on the
dashboard, but then you might run the risk of data not being available in
the datamart, which is why we moved many of the report tools from datamart
to aggregation service some time back. With a more automated and controlled
data mart (coming soon) this becomes a much more attractive option again.
This is part of a more general discussion on on-the-fly data processing
(aggregation service) vs. pre-processed data (data mart) for all our
reporting/presentation tools, and it relates to the type of system that is
deployed, e.g. online servers with many users vs. offline install. We have
talked about making it optional whether to use on-the-fly or datamart for
all report tools as part of a system setting. Going for datamart makes a lot
of sense in multiuser settings with lots of data requests and heavy data
processing needed. We are moving towards a better managed/controlled data
mart where we have scheduled exports that can make sure it is always up to
date (e.g. nightly builds), but still that does not mean instant as in going
directly from data entry to a report/chart.
Ola,----------------------------------
Ola Hodne Titlestad (Mr)
HISP
Department of Informatics
University of OsloMobile: +47 48069736
Home address: Vetlandsvn. 95B, 0685 Oslo, Norway. Googlemaps linkOn 29 November 2010 09:29, Jason Pickering <jason.p.pickering@gmail.com> > wrote:
Created a blueprint here..
https://blueprints.launchpad.net/dhis2/+spec/cached-dashboard-charts
On Mon, Nov 29, 2010 at 9:03 AM, Knut Staring <knutst@gmail.com> wrote:
> Agree. We should rather add one or more "Refresh" buttons.
>
> Knut (from my phone)
>
> On Nov 29, 2010 6:17 AM, "Jason Pickering" <jason.p.pickering@gmail.com> >> > wrote:
>
> Not sure if this a bug or feature, but each time I access the
> Dashboard in 2.0.5, charts that have been added to the dashboard seem
> to be regenerated. The charts involve a fair a bit of aggregation, so
> they take several seconds (minutes actually in one case) to generate.
>
> Is this the intended behaviour?
>
> I would think it would make more sense to use a pre-generated copy for
> the dashboard?
>
>
> Regards,
> Jason
>
>
> --
> Jason P. Pickering
> email: jason.p.pickering@gmail.com
> tel:+260968395190
>
> _______________________________________________
> Mailing list: DHIS 2 developers in Launchpad
> Post to : dhis2-devs@lists.launchpad.net
> Unsubscribe : DHIS 2 developers in Launchpad
> More help : ListHelp - Launchpad Help
>--
Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+260968395190_______________________________________________
Mailing list: DHIS 2 developers in Launchpad
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : DHIS 2 developers in Launchpad
More help : ListHelp - Launchpad Help
--
Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+260968395190