2.29/2.30 WARNING - do not use Maintenance->ClearAnalyticsTables & parameter "-T analytics*" for database dumps

Hi

For as long as I can remember, we have used the standard parameter “-T analytics*” when dumping a DHIS2 database into e.g. a backup or similar.

The purpose of the parameter was to exclude all analytics tables from the dump, since it is significantly faster to restore a dump without analytics tables and then run analytics to re-create them (due to the use of multi-threading), compared to dumping and restoring a database instance with all the analytics table (restore is NOT using multi-threading).

For some reason, in 2.29 a new table that stores periodboundary data for Program Indicators was called “analyticsperiodboundary” - which means the standard pg_dump parameter will leave that table behind together with all other “analytics*” tables.

Furthermore, the routine called “Clear Analytics Tables” found under Data Administration -> Maintenance is as before deleting all tables named Analytics* - THE PROBLEM IS THAT IT ALSO DELETES THE NEW ANALYTICSPERIODBOUNDARY TABLE (same in both 2.29 and 2.30)

Which will crash your system in the sense that you won’t see any program indicator data in dashboards etc.

The “analyticsperiodboundary” table will be re-created and re-populated with DEFAULT (boundless) Program Indicator Period boundaries when you re-start the system (it’s part of the TableAlteror routine during startup), but

  • you have to re-start the system

  • you will lose any non-default boundary settings used for any program indicator.

This has also been reported as a high-priority bug on JIRA (DHIS2-4260).

Regards

Calle

···

Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg


1 Like

Thats nasty alright. I guess using "-T anlytics_*" instead would
help. But there are so many backup scripts out there broken by this
that it will be better to rename the table.

···

On 11 July 2018 at 22:39, Calle Hedberg <calle.hedberg@gmail.com> wrote:

Hi

For as long as I can remember, we have used the standard parameter "-T
analytics*" when dumping a DHIS2 database into e.g. a backup or similar.

The purpose of the parameter was to exclude all analytics tables from the
dump, since it is significantly faster to restore a dump without analytics
tables and then run analytics to re-create them (due to the use of
multi-threading), compared to dumping and restoring a database instance with
all the analytics table (restore is NOT using multi-threading).

For some reason, in 2.29 a new table that stores periodboundary data for
Program Indicators was called "analyticsperiodboundary" - which means the
standard pg_dump parameter will leave that table behind together with all
other "analytics*" tables.

Furthermore, the routine called "Clear Analytics Tables" found under Data
Administration -> Maintenance is as before deleting all tables named
Analytics* - THE PROBLEM IS THAT IT ALSO DELETES THE NEW
ANALYTICSPERIODBOUNDARY TABLE (same in both 2.29 and 2.30)

Which will crash your system in the sense that you won't see any program
indicator data in dashboards etc.

The "analyticsperiodboundary" table will be re-created and re-populated with
DEFAULT (boundless) Program Indicator Period boundaries when you re-start
the system (it's part of the TableAlteror routine during startup), but
- you have to re-start the system
- you will lose any non-default boundary settings used for any program
indicator.

This has also been reported as a high-priority bug on JIRA (DHIS2-4260).

Regards
Calle

*******************************************

Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg

*******************************************

_______________________________________________
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help : https://help.launchpad.net/ListHelp

It would have been good if all the “dispensable” tables could rather have started with an underscore (though that again will of course break extant scripts).

Knut

···

On Thu, Jul 12, 2018, 1:27 PM Bob Jolliffe bobjolliffe@gmail.com wrote:

Thats nasty alright. I guess using “-T anlytics_*” instead would

help. But there are so many backup scripts out there broken by this

that it will be better to rename the table.

On 11 July 2018 at 22:39, Calle Hedberg calle.hedberg@gmail.com wrote:

Hi

For as long as I can remember, we have used the standard parameter "-T

analytics*" when dumping a DHIS2 database into e.g. a backup or similar.

The purpose of the parameter was to exclude all analytics tables from the

dump, since it is significantly faster to restore a dump without analytics

tables and then run analytics to re-create them (due to the use of

multi-threading), compared to dumping and restoring a database instance with

all the analytics table (restore is NOT using multi-threading).

For some reason, in 2.29 a new table that stores periodboundary data for

Program Indicators was called “analyticsperiodboundary” - which means the

standard pg_dump parameter will leave that table behind together with all

other “analytics*” tables.

Furthermore, the routine called “Clear Analytics Tables” found under Data

Administration -> Maintenance is as before deleting all tables named

Analytics* - THE PROBLEM IS THAT IT ALSO DELETES THE NEW

ANALYTICSPERIODBOUNDARY TABLE (same in both 2.29 and 2.30)

Which will crash your system in the sense that you won’t see any program

indicator data in dashboards etc.

The “analyticsperiodboundary” table will be re-created and re-populated with

DEFAULT (boundless) Program Indicator Period boundaries when you re-start

the system (it’s part of the TableAlteror routine during startup), but

  • you have to re-start the system
  • you will lose any non-default boundary settings used for any program

indicator.

This has also been reported as a high-priority bug on JIRA (DHIS2-4260).

Regards

Calle


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg



Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Bob,

No response/action on the JIRA bug report yet - I guess most developers are on leave (wonderful summer here in Norway this year).

Otherwise I agree, the name of that table does not fit the general naming convention as far as I can see. It would make more sense to call it e.g. “programindicator_periodboundary”. The name then provides an intuitive description of the content and it sorts together with the group of programindicator tables.

Regards

calle

···

On 12 July 2018 at 08:57, Bob Jolliffe bobjolliffe@gmail.com wrote:

Thats nasty alright. I guess using “-T anlytics_*” instead would

help. But there are so many backup scripts out there broken by this

that it will be better to rename the table.

On 11 July 2018 at 22:39, Calle Hedberg calle.hedberg@gmail.com wrote:

Hi

For as long as I can remember, we have used the standard parameter "-T

analytics*" when dumping a DHIS2 database into e.g. a backup or similar.

The purpose of the parameter was to exclude all analytics tables from the

dump, since it is significantly faster to restore a dump without analytics

tables and then run analytics to re-create them (due to the use of

multi-threading), compared to dumping and restoring a database instance with

all the analytics table (restore is NOT using multi-threading).

For some reason, in 2.29 a new table that stores periodboundary data for

Program Indicators was called “analyticsperiodboundary” - which means the

standard pg_dump parameter will leave that table behind together with all

other “analytics*” tables.

Furthermore, the routine called “Clear Analytics Tables” found under Data

Administration -> Maintenance is as before deleting all tables named

Analytics* - THE PROBLEM IS THAT IT ALSO DELETES THE NEW

ANALYTICSPERIODBOUNDARY TABLE (same in both 2.29 and 2.30)

Which will crash your system in the sense that you won’t see any program

indicator data in dashboards etc.

The “analyticsperiodboundary” table will be re-created and re-populated with

DEFAULT (boundless) Program Indicator Period boundaries when you re-start

the system (it’s part of the TableAlteror routine during startup), but

  • you have to re-start the system
  • you will lose any non-default boundary settings used for any program

indicator.

This has also been reported as a high-priority bug on JIRA (DHIS2-4260).

Regards

Calle


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg



Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg


Hi Calle

We are aware of this issue (actually it caused a problem with us in Lao), for now… I would also say don’t use this functionality, its better to manually delete the analytics_* tables if you need it. We will rename that table soon we hope, and that should fix it (this also causes potential issues with your backups…)

···

On Thu, Jul 12, 2018 at 3:58 PM, Calle Hedberg calle.hedberg@gmail.com wrote:

Bob,

No response/action on the JIRA bug report yet - I guess most developers are on leave (wonderful summer here in Norway this year).

Otherwise I agree, the name of that table does not fit the general naming convention as far as I can see. It would make more sense to call it e.g. “programindicator_periodboundary”. The name then provides an intuitive description of the content and it sorts together with the group of programindicator tables.

Regards

calle


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On 12 July 2018 at 08:57, Bob Jolliffe bobjolliffe@gmail.com wrote:

Thats nasty alright. I guess using “-T anlytics_*” instead would

help. But there are so many backup scripts out there broken by this

that it will be better to rename the table.

On 11 July 2018 at 22:39, Calle Hedberg calle.hedberg@gmail.com wrote:

Hi

For as long as I can remember, we have used the standard parameter "-T

analytics*" when dumping a DHIS2 database into e.g. a backup or similar.

The purpose of the parameter was to exclude all analytics tables from the

dump, since it is significantly faster to restore a dump without analytics

tables and then run analytics to re-create them (due to the use of

multi-threading), compared to dumping and restoring a database instance with

all the analytics table (restore is NOT using multi-threading).

For some reason, in 2.29 a new table that stores periodboundary data for

Program Indicators was called “analyticsperiodboundary” - which means the

standard pg_dump parameter will leave that table behind together with all

other “analytics*” tables.

Furthermore, the routine called “Clear Analytics Tables” found under Data

Administration -> Maintenance is as before deleting all tables named

Analytics* - THE PROBLEM IS THAT IT ALSO DELETES THE NEW

ANALYTICSPERIODBOUNDARY TABLE (same in both 2.29 and 2.30)

Which will crash your system in the sense that you won’t see any program

indicator data in dashboards etc.

The “analyticsperiodboundary” table will be re-created and re-populated with

DEFAULT (boundless) Program Indicator Period boundaries when you re-start

the system (it’s part of the TableAlteror routine during startup), but

  • you have to re-start the system
  • you will lose any non-default boundary settings used for any program

indicator.

This has also been reported as a high-priority bug on JIRA (DHIS2-4260).

Regards

Calle


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg



Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg


Hi Morten, sorry but which functionality are you suggesting should not be used? What do you mean by manually deleting?

Thanks,

Knut

···

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Thu, Jul 12, 2018 at 3:58 PM, Calle Hedberg calle.hedberg@gmail.com wrote:

Bob,

No response/action on the JIRA bug report yet - I guess most developers are on leave (wonderful summer here in Norway this year).

Otherwise I agree, the name of that table does not fit the general naming convention as far as I can see. It would make more sense to call it e.g. “programindicator_periodboundary”. The name then provides an intuitive description of the content and it sorts together with the group of programindicator tables.

Regards

calle


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

On 12 July 2018 at 08:57, Bob Jolliffe bobjolliffe@gmail.com wrote:

Thats nasty alright. I guess using “-T anlytics_*” instead would

help. But there are so many backup scripts out there broken by this

that it will be better to rename the table.

On 11 July 2018 at 22:39, Calle Hedberg calle.hedberg@gmail.com wrote:

Hi

For as long as I can remember, we have used the standard parameter "-T

analytics*" when dumping a DHIS2 database into e.g. a backup or similar.

The purpose of the parameter was to exclude all analytics tables from the

dump, since it is significantly faster to restore a dump without analytics

tables and then run analytics to re-create them (due to the use of

multi-threading), compared to dumping and restoring a database instance with

all the analytics table (restore is NOT using multi-threading).

For some reason, in 2.29 a new table that stores periodboundary data for

Program Indicators was called “analyticsperiodboundary” - which means the

standard pg_dump parameter will leave that table behind together with all

other “analytics*” tables.

Furthermore, the routine called “Clear Analytics Tables” found under Data

Administration -> Maintenance is as before deleting all tables named

Analytics* - THE PROBLEM IS THAT IT ALSO DELETES THE NEW

ANALYTICSPERIODBOUNDARY TABLE (same in both 2.29 and 2.30)

Which will crash your system in the sense that you won’t see any program

indicator data in dashboards etc.

The “analyticsperiodboundary” table will be re-created and re-populated with

DEFAULT (boundless) Program Indicator Period boundaries when you re-start

the system (it’s part of the TableAlteror routine during startup), but

  • you have to re-start the system
  • you will lose any non-default boundary settings used for any program

indicator.

This has also been reported as a high-priority bug on JIRA (DHIS2-4260).

Regards

Calle


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg



Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg


Hi Knut

I mean the main issue the thread was about… using maintenance => clear analytic tables, it will delete analytics* which includes the analytical boundaries.

···

On Thu, Jul 12, 2018 at 5:11 PM, Knut Staring knutst@gmail.com wrote:

Hi Morten, sorry but which functionality are you suggesting should not be used? What do you mean by manually deleting?

Thanks,

Knut

On Thu, Jul 12, 2018, 4:23 PM Morten Olav Hansen morten@dhis2.org wrote:

Hi Calle

We are aware of this issue (actually it caused a problem with us in Lao), for now… I would also say don’t use this functionality, its better to manually delete the analytics_* tables if you need it. We will rename that table soon we hope, and that should fix it (this also causes potential issues with your backups…)


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Thu, Jul 12, 2018 at 3:58 PM, Calle Hedberg calle.hedberg@gmail.com wrote:

Bob,

No response/action on the JIRA bug report yet - I guess most developers are on leave (wonderful summer here in Norway this year).

Otherwise I agree, the name of that table does not fit the general naming convention as far as I can see. It would make more sense to call it e.g. “programindicator_periodboundary”. The name then provides an intuitive description of the content and it sorts together with the group of programindicator tables.

Regards

calle


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

On 12 July 2018 at 08:57, Bob Jolliffe bobjolliffe@gmail.com wrote:

Thats nasty alright. I guess using “-T anlytics_*” instead would

help. But there are so many backup scripts out there broken by this

that it will be better to rename the table.

On 11 July 2018 at 22:39, Calle Hedberg calle.hedberg@gmail.com wrote:

Hi

For as long as I can remember, we have used the standard parameter "-T

analytics*" when dumping a DHIS2 database into e.g. a backup or similar.

The purpose of the parameter was to exclude all analytics tables from the

dump, since it is significantly faster to restore a dump without analytics

tables and then run analytics to re-create them (due to the use of

multi-threading), compared to dumping and restoring a database instance with

all the analytics table (restore is NOT using multi-threading).

For some reason, in 2.29 a new table that stores periodboundary data for

Program Indicators was called “analyticsperiodboundary” - which means the

standard pg_dump parameter will leave that table behind together with all

other “analytics*” tables.

Furthermore, the routine called “Clear Analytics Tables” found under Data

Administration -> Maintenance is as before deleting all tables named

Analytics* - THE PROBLEM IS THAT IT ALSO DELETES THE NEW

ANALYTICSPERIODBOUNDARY TABLE (same in both 2.29 and 2.30)

Which will crash your system in the sense that you won’t see any program

indicator data in dashboards etc.

The “analyticsperiodboundary” table will be re-created and re-populated with

DEFAULT (boundless) Program Indicator Period boundaries when you re-start

the system (it’s part of the TableAlteror routine during startup), but

  • you have to re-start the system
  • you will lose any non-default boundary settings used for any program

indicator.

This has also been reported as a high-priority bug on JIRA (DHIS2-4260).

Regards

Calle


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg



Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg


Right… I assume you could trigger that also through the API in order to continue to have backup scripts without manual intervention?

···

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Thu, Jul 12, 2018 at 5:11 PM, Knut Staring knutst@gmail.com wrote:

Hi Morten, sorry but which functionality are you suggesting should not be used? What do you mean by manually deleting?

Thanks,

Knut

On Thu, Jul 12, 2018, 4:23 PM Morten Olav Hansen morten@dhis2.org wrote:

Hi Calle

We are aware of this issue (actually it caused a problem with us in Lao), for now… I would also say don’t use this functionality, its better to manually delete the analytics_* tables if you need it. We will rename that table soon we hope, and that should fix it (this also causes potential issues with your backups…)


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Thu, Jul 12, 2018 at 3:58 PM, Calle Hedberg calle.hedberg@gmail.com wrote:

Bob,

No response/action on the JIRA bug report yet - I guess most developers are on leave (wonderful summer here in Norway this year).

Otherwise I agree, the name of that table does not fit the general naming convention as far as I can see. It would make more sense to call it e.g. “programindicator_periodboundary”. The name then provides an intuitive description of the content and it sorts together with the group of programindicator tables.

Regards

calle


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

On 12 July 2018 at 08:57, Bob Jolliffe bobjolliffe@gmail.com wrote:

Thats nasty alright. I guess using “-T anlytics_*” instead would

help. But there are so many backup scripts out there broken by this

that it will be better to rename the table.

On 11 July 2018 at 22:39, Calle Hedberg calle.hedberg@gmail.com wrote:

Hi

For as long as I can remember, we have used the standard parameter "-T

analytics*" when dumping a DHIS2 database into e.g. a backup or similar.

The purpose of the parameter was to exclude all analytics tables from the

dump, since it is significantly faster to restore a dump without analytics

tables and then run analytics to re-create them (due to the use of

multi-threading), compared to dumping and restoring a database instance with

all the analytics table (restore is NOT using multi-threading).

For some reason, in 2.29 a new table that stores periodboundary data for

Program Indicators was called “analyticsperiodboundary” - which means the

standard pg_dump parameter will leave that table behind together with all

other “analytics*” tables.

Furthermore, the routine called “Clear Analytics Tables” found under Data

Administration -> Maintenance is as before deleting all tables named

Analytics* - THE PROBLEM IS THAT IT ALSO DELETES THE NEW

ANALYTICSPERIODBOUNDARY TABLE (same in both 2.29 and 2.30)

Which will crash your system in the sense that you won’t see any program

indicator data in dashboards etc.

The “analyticsperiodboundary” table will be re-created and re-populated with

DEFAULT (boundless) Program Indicator Period boundaries when you re-start

the system (it’s part of the TableAlteror routine during startup), but

  • you have to re-start the system
  • you will lose any non-default boundary settings used for any program

indicator.

This has also been reported as a high-priority bug on JIRA (DHIS2-4260).

Regards

Calle


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg



Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg


I have started the work on renaming the table. I will update this thread as soon as I make progress.

···

On Thu, Jul 12, 2018 at 12:17 PM, Morten Olav Hansen morten@dhis2.org wrote:

Hi Knut

I mean the main issue the thread was about… using maintenance => clear analytic tables, it will delete analytics* which includes the analytical boundaries.


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Thu, Jul 12, 2018 at 5:11 PM, Knut Staring knutst@gmail.com wrote:

Hi Morten, sorry but which functionality are you suggesting should not be used? What do you mean by manually deleting?

Thanks,

Knut

On Thu, Jul 12, 2018, 4:23 PM Morten Olav Hansen morten@dhis2.org wrote:

Hi Calle

We are aware of this issue (actually it caused a problem with us in Lao), for now… I would also say don’t use this functionality, its better to manually delete the analytics_* tables if you need it. We will rename that table soon we hope, and that should fix it (this also causes potential issues with your backups…)


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Thu, Jul 12, 2018 at 3:58 PM, Calle Hedberg calle.hedberg@gmail.com wrote:

Bob,

No response/action on the JIRA bug report yet - I guess most developers are on leave (wonderful summer here in Norway this year).

Otherwise I agree, the name of that table does not fit the general naming convention as far as I can see. It would make more sense to call it e.g. “programindicator_periodboundary”. The name then provides an intuitive description of the content and it sorts together with the group of programindicator tables.

Regards

calle


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

On 12 July 2018 at 08:57, Bob Jolliffe bobjolliffe@gmail.com wrote:

Thats nasty alright. I guess using “-T anlytics_*” instead would

help. But there are so many backup scripts out there broken by this

that it will be better to rename the table.

On 11 July 2018 at 22:39, Calle Hedberg calle.hedberg@gmail.com wrote:

Hi

For as long as I can remember, we have used the standard parameter "-T

analytics*" when dumping a DHIS2 database into e.g. a backup or similar.

The purpose of the parameter was to exclude all analytics tables from the

dump, since it is significantly faster to restore a dump without analytics

tables and then run analytics to re-create them (due to the use of

multi-threading), compared to dumping and restoring a database instance with

all the analytics table (restore is NOT using multi-threading).

For some reason, in 2.29 a new table that stores periodboundary data for

Program Indicators was called “analyticsperiodboundary” - which means the

standard pg_dump parameter will leave that table behind together with all

other “analytics*” tables.

Furthermore, the routine called “Clear Analytics Tables” found under Data

Administration -> Maintenance is as before deleting all tables named

Analytics* - THE PROBLEM IS THAT IT ALSO DELETES THE NEW

ANALYTICSPERIODBOUNDARY TABLE (same in both 2.29 and 2.30)

Which will crash your system in the sense that you won’t see any program

indicator data in dashboards etc.

The “analyticsperiodboundary” table will be re-created and re-populated with

DEFAULT (boundless) Program Indicator Period boundaries when you re-start

the system (it’s part of the TableAlteror routine during startup), but

  • you have to re-start the system
  • you will lose any non-default boundary settings used for any program

indicator.

This has also been reported as a high-priority bug on JIRA (DHIS2-4260).

Regards

Calle


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg



Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg


Stian Sandvold
Software developer, DHIS2

University of Oslo

http://www.dhis2.org

Ok, thanks Stian (no need to work on this then Viet)

(knut, using pg_dump we normally filter away analytic* which means no period boundaries…)

···

On Thu, Jul 12, 2018 at 6:14 PM, Stian Sandvold stian@dhis2.org wrote:

I have started the work on renaming the table. I will update this thread as soon as I make progress.

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Thu, Jul 12, 2018 at 12:17 PM, Morten Olav Hansen morten@dhis2.org wrote:

Hi Knut

I mean the main issue the thread was about… using maintenance => clear analytic tables, it will delete analytics* which includes the analytical boundaries.


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Software developer, DHIS2

Stian Sandvold
University of Oslo

http://www.dhis2.org

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Thu, Jul 12, 2018 at 5:11 PM, Knut Staring knutst@gmail.com wrote:

Hi Morten, sorry but which functionality are you suggesting should not be used? What do you mean by manually deleting?

Thanks,

Knut

On Thu, Jul 12, 2018, 4:23 PM Morten Olav Hansen morten@dhis2.org wrote:

Hi Calle

We are aware of this issue (actually it caused a problem with us in Lao), for now… I would also say don’t use this functionality, its better to manually delete the analytics_* tables if you need it. We will rename that table soon we hope, and that should fix it (this also causes potential issues with your backups…)


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Thu, Jul 12, 2018 at 3:58 PM, Calle Hedberg calle.hedberg@gmail.com wrote:

Bob,

No response/action on the JIRA bug report yet - I guess most developers are on leave (wonderful summer here in Norway this year).

Otherwise I agree, the name of that table does not fit the general naming convention as far as I can see. It would make more sense to call it e.g. “programindicator_periodboundary”. The name then provides an intuitive description of the content and it sorts together with the group of programindicator tables.

Regards

calle


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

On 12 July 2018 at 08:57, Bob Jolliffe bobjolliffe@gmail.com wrote:

Thats nasty alright. I guess using “-T anlytics_*” instead would

help. But there are so many backup scripts out there broken by this

that it will be better to rename the table.

On 11 July 2018 at 22:39, Calle Hedberg calle.hedberg@gmail.com wrote:

Hi

For as long as I can remember, we have used the standard parameter "-T

analytics*" when dumping a DHIS2 database into e.g. a backup or similar.

The purpose of the parameter was to exclude all analytics tables from the

dump, since it is significantly faster to restore a dump without analytics

tables and then run analytics to re-create them (due to the use of

multi-threading), compared to dumping and restoring a database instance with

all the analytics table (restore is NOT using multi-threading).

For some reason, in 2.29 a new table that stores periodboundary data for

Program Indicators was called “analyticsperiodboundary” - which means the

standard pg_dump parameter will leave that table behind together with all

other “analytics*” tables.

Furthermore, the routine called “Clear Analytics Tables” found under Data

Administration -> Maintenance is as before deleting all tables named

Analytics* - THE PROBLEM IS THAT IT ALSO DELETES THE NEW

ANALYTICSPERIODBOUNDARY TABLE (same in both 2.29 and 2.30)

Which will crash your system in the sense that you won’t see any program

indicator data in dashboards etc.

The “analyticsperiodboundary” table will be re-created and re-populated with

DEFAULT (boundless) Program Indicator Period boundaries when you re-start

the system (it’s part of the TableAlteror routine during startup), but

  • you have to re-start the system
  • you will lose any non-default boundary settings used for any program

indicator.

This has also been reported as a high-priority bug on JIRA (DHIS2-4260).

Regards

Calle


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg



Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg


Hi

I’m glad it’s being addressed - but I am less happy to hear you are aware of it but haven’t said anything to the community… These type of bugs causes havoc, and waste a lot of time for users affected while they try to figure out what’s gone wrong.

There are two major lessons here:

Firstly, to stick to a standard and manageable naming convention for both tables and fields (and interfaces).

Secondly, to inform the user community promptly when you become aware of major, “showstopper”, bugs.

Best regards

calle

···

On 12 July 2018 at 13:15, Morten Olav Hansen morten@dhis2.org wrote:

Ok, thanks Stian (no need to work on this then Viet)

(knut, using pg_dump we normally filter away analytic* which means no period boundaries…)


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Thu, Jul 12, 2018 at 6:14 PM, Stian Sandvold stian@dhis2.org wrote:

I have started the work on renaming the table. I will update this thread as soon as I make progress.

On Thu, Jul 12, 2018 at 12:17 PM, Morten Olav Hansen morten@dhis2.org wrote:

Hi Knut

I mean the main issue the thread was about… using maintenance => clear analytic tables, it will delete analytics* which includes the analytical boundaries.


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Software developer, DHIS2

Stian Sandvold
University of Oslo

http://www.dhis2.org

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Thu, Jul 12, 2018 at 5:11 PM, Knut Staring knutst@gmail.com wrote:

Hi Morten, sorry but which functionality are you suggesting should not be used? What do you mean by manually deleting?

Thanks,

Knut

On Thu, Jul 12, 2018, 4:23 PM Morten Olav Hansen morten@dhis2.org wrote:

Hi Calle

We are aware of this issue (actually it caused a problem with us in Lao), for now… I would also say don’t use this functionality, its better to manually delete the analytics_* tables if you need it. We will rename that table soon we hope, and that should fix it (this also causes potential issues with your backups…)


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Thu, Jul 12, 2018 at 3:58 PM, Calle Hedberg calle.hedberg@gmail.com wrote:

Bob,

No response/action on the JIRA bug report yet - I guess most developers are on leave (wonderful summer here in Norway this year).

Otherwise I agree, the name of that table does not fit the general naming convention as far as I can see. It would make more sense to call it e.g. “programindicator_periodboundary”. The name then provides an intuitive description of the content and it sorts together with the group of programindicator tables.

Regards

calle


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

On 12 July 2018 at 08:57, Bob Jolliffe bobjolliffe@gmail.com wrote:

Thats nasty alright. I guess using “-T anlytics_*” instead would

help. But there are so many backup scripts out there broken by this

that it will be better to rename the table.

On 11 July 2018 at 22:39, Calle Hedberg calle.hedberg@gmail.com wrote:

Hi

For as long as I can remember, we have used the standard parameter "-T

analytics*" when dumping a DHIS2 database into e.g. a backup or similar.

The purpose of the parameter was to exclude all analytics tables from the

dump, since it is significantly faster to restore a dump without analytics

tables and then run analytics to re-create them (due to the use of

multi-threading), compared to dumping and restoring a database instance with

all the analytics table (restore is NOT using multi-threading).

For some reason, in 2.29 a new table that stores periodboundary data for

Program Indicators was called “analyticsperiodboundary” - which means the

standard pg_dump parameter will leave that table behind together with all

other “analytics*” tables.

Furthermore, the routine called “Clear Analytics Tables” found under Data

Administration -> Maintenance is as before deleting all tables named

Analytics* - THE PROBLEM IS THAT IT ALSO DELETES THE NEW

ANALYTICSPERIODBOUNDARY TABLE (same in both 2.29 and 2.30)

Which will crash your system in the sense that you won’t see any program

indicator data in dashboards etc.

The “analyticsperiodboundary” table will be re-created and re-populated with

DEFAULT (boundless) Program Indicator Period boundaries when you re-start

the system (it’s part of the TableAlteror routine during startup), but

  • you have to re-start the system
  • you will lose any non-default boundary settings used for any program

indicator.

This has also been reported as a high-priority bug on JIRA (DHIS2-4260).

Regards

Calle


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg



Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg



Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg


Hi Calle

I have only been aware of this bug for a few days (when it hit us in Lao), and then we have both release and summer holidays going on. I agree the response is not ideal, but its not the easiest time for fixing bug issues.

We are working hard on fixing this now (it might take 1-2 days), and at that time we will recommend that everyone updates their 229 and 230 instances (I will leave that up to Stian when he is ready)

This was a design issue, and they are not that easy to fix (without causing havoc internally), but hopefully we have learned from this and we won’t name domain tables analytics* anymore.

···

Morten Olav Hansen

Senior Engineer, DHIS 2

Team Integration Lead

University of Oslo

http://www.dhis2.org

On Thu, Jul 12, 2018 at 6:33 PM, Calle Hedberg calle.hedberg@gmail.com wrote:

Hi

I’m glad it’s being addressed - but I am less happy to hear you are aware of it but haven’t said anything to the community… These type of bugs causes havoc, and waste a lot of time for users affected while they try to figure out what’s gone wrong.

There are two major lessons here:

Firstly, to stick to a standard and manageable naming convention for both tables and fields (and interfaces).

Secondly, to inform the user community promptly when you become aware of major, “showstopper”, bugs.

Best regards

calle

On 12 July 2018 at 13:15, Morten Olav Hansen morten@dhis2.org wrote:

Ok, thanks Stian (no need to work on this then Viet)

(knut, using pg_dump we normally filter away analytic* which means no period boundaries…)


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg


Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Thu, Jul 12, 2018 at 6:14 PM, Stian Sandvold stian@dhis2.org wrote:

I have started the work on renaming the table. I will update this thread as soon as I make progress.

On Thu, Jul 12, 2018 at 12:17 PM, Morten Olav Hansen morten@dhis2.org wrote:

Hi Knut

I mean the main issue the thread was about… using maintenance => clear analytic tables, it will delete analytics* which includes the analytical boundaries.


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Software developer, DHIS2

Stian Sandvold
University of Oslo

http://www.dhis2.org

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Thu, Jul 12, 2018 at 5:11 PM, Knut Staring knutst@gmail.com wrote:

Hi Morten, sorry but which functionality are you suggesting should not be used? What do you mean by manually deleting?

Thanks,

Knut

On Thu, Jul 12, 2018, 4:23 PM Morten Olav Hansen morten@dhis2.org wrote:

Hi Calle

We are aware of this issue (actually it caused a problem with us in Lao), for now… I would also say don’t use this functionality, its better to manually delete the analytics_* tables if you need it. We will rename that table soon we hope, and that should fix it (this also causes potential issues with your backups…)


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Thu, Jul 12, 2018 at 3:58 PM, Calle Hedberg calle.hedberg@gmail.com wrote:

Bob,

No response/action on the JIRA bug report yet - I guess most developers are on leave (wonderful summer here in Norway this year).

Otherwise I agree, the name of that table does not fit the general naming convention as far as I can see. It would make more sense to call it e.g. “programindicator_periodboundary”. The name then provides an intuitive description of the content and it sorts together with the group of programindicator tables.

Regards

calle


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

On 12 July 2018 at 08:57, Bob Jolliffe bobjolliffe@gmail.com wrote:

Thats nasty alright. I guess using “-T anlytics_*” instead would

help. But there are so many backup scripts out there broken by this

that it will be better to rename the table.

On 11 July 2018 at 22:39, Calle Hedberg calle.hedberg@gmail.com wrote:

Hi

For as long as I can remember, we have used the standard parameter "-T

analytics*" when dumping a DHIS2 database into e.g. a backup or similar.

The purpose of the parameter was to exclude all analytics tables from the

dump, since it is significantly faster to restore a dump without analytics

tables and then run analytics to re-create them (due to the use of

multi-threading), compared to dumping and restoring a database instance with

all the analytics table (restore is NOT using multi-threading).

For some reason, in 2.29 a new table that stores periodboundary data for

Program Indicators was called “analyticsperiodboundary” - which means the

standard pg_dump parameter will leave that table behind together with all

other “analytics*” tables.

Furthermore, the routine called “Clear Analytics Tables” found under Data

Administration -> Maintenance is as before deleting all tables named

Analytics* - THE PROBLEM IS THAT IT ALSO DELETES THE NEW

ANALYTICSPERIODBOUNDARY TABLE (same in both 2.29 and 2.30)

Which will crash your system in the sense that you won’t see any program

indicator data in dashboards etc.

The “analyticsperiodboundary” table will be re-created and re-populated with

DEFAULT (boundless) Program Indicator Period boundaries when you re-start

the system (it’s part of the TableAlteror routine during startup), but

  • you have to re-start the system
  • you will lose any non-default boundary settings used for any program

indicator.

This has also been reported as a high-priority bug on JIRA (DHIS2-4260).

Regards

Calle


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg



Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg


Would be great if the convention could be that ALL generated tables (including analytics) started with an underscore. This convention is sort of already in place, just not applied consistently.

Knut

···

On 12 July 2018 at 13:15, Morten Olav Hansen morten@dhis2.org wrote:

Ok, thanks Stian (no need to work on this then Viet)

(knut, using pg_dump we normally filter away analytic* which means no period boundaries…)


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg


Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Thu, Jul 12, 2018 at 6:14 PM, Stian Sandvold stian@dhis2.org wrote:

I have started the work on renaming the table. I will update this thread as soon as I make progress.

On Thu, Jul 12, 2018 at 12:17 PM, Morten Olav Hansen morten@dhis2.org wrote:

Hi Knut

I mean the main issue the thread was about… using maintenance => clear analytic tables, it will delete analytics* which includes the analytical boundaries.


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Software developer, DHIS2

Stian Sandvold
University of Oslo

http://www.dhis2.org

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Thu, Jul 12, 2018 at 5:11 PM, Knut Staring knutst@gmail.com wrote:

Hi Morten, sorry but which functionality are you suggesting should not be used? What do you mean by manually deleting?

Thanks,

Knut

On Thu, Jul 12, 2018, 4:23 PM Morten Olav Hansen morten@dhis2.org wrote:

Hi Calle

We are aware of this issue (actually it caused a problem with us in Lao), for now… I would also say don’t use this functionality, its better to manually delete the analytics_* tables if you need it. We will rename that table soon we hope, and that should fix it (this also causes potential issues with your backups…)


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Thu, Jul 12, 2018 at 3:58 PM, Calle Hedberg calle.hedberg@gmail.com wrote:

Bob,

No response/action on the JIRA bug report yet - I guess most developers are on leave (wonderful summer here in Norway this year).

Otherwise I agree, the name of that table does not fit the general naming convention as far as I can see. It would make more sense to call it e.g. “programindicator_periodboundary”. The name then provides an intuitive description of the content and it sorts together with the group of programindicator tables.

Regards

calle


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

On 12 July 2018 at 08:57, Bob Jolliffe bobjolliffe@gmail.com wrote:

Thats nasty alright. I guess using “-T anlytics_*” instead would

help. But there are so many backup scripts out there broken by this

that it will be better to rename the table.

On 11 July 2018 at 22:39, Calle Hedberg calle.hedberg@gmail.com wrote:

Hi

For as long as I can remember, we have used the standard parameter "-T

analytics*" when dumping a DHIS2 database into e.g. a backup or similar.

The purpose of the parameter was to exclude all analytics tables from the

dump, since it is significantly faster to restore a dump without analytics

tables and then run analytics to re-create them (due to the use of

multi-threading), compared to dumping and restoring a database instance with

all the analytics table (restore is NOT using multi-threading).

For some reason, in 2.29 a new table that stores periodboundary data for

Program Indicators was called “analyticsperiodboundary” - which means the

standard pg_dump parameter will leave that table behind together with all

other “analytics*” tables.

Furthermore, the routine called “Clear Analytics Tables” found under Data

Administration -> Maintenance is as before deleting all tables named

Analytics* - THE PROBLEM IS THAT IT ALSO DELETES THE NEW

ANALYTICSPERIODBOUNDARY TABLE (same in both 2.29 and 2.30)

Which will crash your system in the sense that you won’t see any program

indicator data in dashboards etc.

The “analyticsperiodboundary” table will be re-created and re-populated with

DEFAULT (boundless) Program Indicator Period boundaries when you re-start

the system (it’s part of the TableAlteror routine during startup), but

  • you have to re-start the system
  • you will lose any non-default boundary settings used for any program

indicator.

This has also been reported as a high-priority bug on JIRA (DHIS2-4260).

Regards

Calle


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg



Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg


Knut Staring

Department of Information, Evidence and Research
World Health Organization, Geneva, Switzerland
Office: +41 22 791 3683 Mob1: +33 6 4434 2931 Mob2: +47 9188 0522
Skype: knutstar

Agree that might have made sense at the start. But I think the
priority now should be to rename that one table so that all the backup
scripts in the wild get unbroken. Not to break them even more :slight_smile:

···

On 12 July 2018 at 13:02, Knut Staring <knutst@gmail.com> wrote:

Would be great if the convention could be that ALL generated tables
(including analytics) started with an underscore. This convention is sort of
already in place, just not applied consistently.

Knut

On Thu, Jul 12, 2018 at 6:05 PM Calle Hedberg <calle.hedberg@gmail.com> > wrote:

Hi

I'm glad it's being addressed - but I am less happy to hear you are aware
of it but haven't said anything to the community... These type of bugs
causes havoc, and waste a lot of time for users affected while they try to
figure out what's gone wrong.

There are two major lessons here:

Firstly, to stick to a standard and manageable naming convention for both
tables and fields (and interfaces).

Secondly, to inform the user community promptly when you become aware of
major, "showstopper", bugs.

Best regards
calle

On 12 July 2018 at 13:15, Morten Olav Hansen <morten@dhis2.org> wrote:

Ok, thanks Stian (no need to work on this then Viet)

(knut, using pg_dump we normally filter away analytic* which means no
period boundaries..)

--
Morten Olav Hansen
Senior Engineer, DHIS 2
University of Oslo
http://www.dhis2.org

On Thu, Jul 12, 2018 at 6:14 PM, Stian Sandvold <stian@dhis2.org> wrote:

I have started the work on renaming the table. I will update this thread
as soon as I make progress.

On Thu, Jul 12, 2018 at 12:17 PM, Morten Olav Hansen <morten@dhis2.org> >>>> wrote:

Hi Knut

I mean the main issue the thread was about... using maintenance =>
clear analytic tables, it will delete analytics* which includes the
analytical boundaries.

--
Morten Olav Hansen
Senior Engineer, DHIS 2
University of Oslo
http://www.dhis2.org

On Thu, Jul 12, 2018 at 5:11 PM, Knut Staring <knutst@gmail.com> wrote:

Hi Morten, sorry but which functionality are you suggesting should not
be used? What do you mean by manually deleting?

Thanks,
Knut

On Thu, Jul 12, 2018, 4:23 PM Morten Olav Hansen <morten@dhis2.org> >>>>>> wrote:

Hi Calle

We are aware of this issue (actually it caused a problem with us in
Lao), for now.. I would also say don't use this functionality, its better to
manually delete the analytics_* tables if you need it. We will rename that
table soon we hope, and that should fix it (this also causes potential
issues with your backups...)

--
Morten Olav Hansen
Senior Engineer, DHIS 2
University of Oslo
http://www.dhis2.org

On Thu, Jul 12, 2018 at 3:58 PM, Calle Hedberg >>>>>>> <calle.hedberg@gmail.com> wrote:

Bob,

No response/action on the JIRA bug report yet - I guess most
developers are on leave (wonderful summer here in Norway this year).

Otherwise I agree, the name of that table does not fit the general
naming convention as far as I can see. It would make more sense to call it
e.g. "programindicator_periodboundary". The name then provides an intuitive
description of the content and it sorts together with the group of
programindicator tables.

Regards
calle

On 12 July 2018 at 08:57, Bob Jolliffe <bobjolliffe@gmail.com> >>>>>>>> wrote:

Thats nasty alright. I guess using "-T anlytics_*" instead would
help. But there are so many backup scripts out there broken by
this
that it will be better to rename the table.

On 11 July 2018 at 22:39, Calle Hedberg <calle.hedberg@gmail.com> >>>>>>>>> wrote:
> Hi
>
> For as long as I can remember, we have used the standard
> parameter "-T
> analytics*" when dumping a DHIS2 database into e.g. a backup or
> similar.
>
> The purpose of the parameter was to exclude all analytics tables
> from the
> dump, since it is significantly faster to restore a dump without
> analytics
> tables and then run analytics to re-create them (due to the use
> of
> multi-threading), compared to dumping and restoring a database
> instance with
> all the analytics table (restore is NOT using multi-threading).
>
> For some reason, in 2.29 a new table that stores periodboundary
> data for
> Program Indicators was called "analyticsperiodboundary" - which
> means the
> standard pg_dump parameter will leave that table behind together
> with all
> other "analytics*" tables.
>
> Furthermore, the routine called "Clear Analytics Tables" found
> under Data
> Administration -> Maintenance is as before deleting all tables
> named
> Analytics* - THE PROBLEM IS THAT IT ALSO DELETES THE NEW
> ANALYTICSPERIODBOUNDARY TABLE (same in both 2.29 and 2.30)
>
> Which will crash your system in the sense that you won't see any
> program
> indicator data in dashboards etc.
>
> The "analyticsperiodboundary" table will be re-created and
> re-populated with
> DEFAULT (boundless) Program Indicator Period boundaries when you
> re-start
> the system (it's part of the TableAlteror routine during
> startup), but
> - you have to re-start the system
> - you will lose any non-default boundary settings used for any
> program
> indicator.
>
> This has also been reported as a high-priority bug on JIRA
> (DHIS2-4260).
>
> Regards
> Calle
>
> *******************************************
>
> Calle Hedberg
>
> 46D Alma Road, 7700 Rosebank, SOUTH AFRICA
>
> Tel/fax (home): +27-21-685-6472
>
> Cell: +27-82-853-5352
>
> Iridium SatPhone: +8816-315-19119
>
> Email: calle.hedberg@gmail.com
>
> Skype: calle_hedberg
>
> *******************************************
>
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~dhis2-devs
> Post to : dhis2-devs@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~dhis2-devs
> More help : https://help.launchpad.net/ListHelp
>

--

*******************************************

Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg

*******************************************

_______________________________________________
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help : https://help.launchpad.net/ListHelp

_______________________________________________
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help : https://help.launchpad.net/ListHelp

_______________________________________________
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help : https://help.launchpad.net/ListHelp

--
Stian Sandvold
Software developer, DHIS2
University of Oslo
http://www.dhis2.org

_______________________________________________
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help : https://help.launchpad.net/ListHelp

--

*******************************************

Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg

*******************************************

_______________________________________________
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help : https://help.launchpad.net/ListHelp

--
Knut Staring

Department of Information, Evidence and Research
World Health Organization, Geneva, Switzerland
Office: +41 22 791 3683 Mob1: +33 6 4434 2931 Mob2: +47 9188 0522
Skype: knutstar

_______________________________________________
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help : https://help.launchpad.net/ListHelp

The convention today is that resource tables are prefixed with an underscore, while analytics tables start with analytics*. As previously mentioned in this thread, changing this now will probably cause a lot of issues for existing scripts, so I think for now we should just avoid using tables starting with analytics*.

···

On Thu, Jul 12, 2018 at 3:26 PM, Bob Jolliffe bobjolliffe@gmail.com wrote:

Agree that might have made sense at the start. But I think the

priority now should be to rename that one table so that all the backup

scripts in the wild get unbroken. Not to break them even more :slight_smile:

On 12 July 2018 at 13:02, Knut Staring knutst@gmail.com wrote:

Would be great if the convention could be that ALL generated tables

(including analytics) started with an underscore. This convention is sort of

already in place, just not applied consistently.

Knut

On Thu, Jul 12, 2018 at 6:05 PM Calle Hedberg calle.hedberg@gmail.com

wrote:

Hi

I’m glad it’s being addressed - but I am less happy to hear you are aware

of it but haven’t said anything to the community… These type of bugs

causes havoc, and waste a lot of time for users affected while they try to

figure out what’s gone wrong.

There are two major lessons here:

Firstly, to stick to a standard and manageable naming convention for both

tables and fields (and interfaces).

Secondly, to inform the user community promptly when you become aware of

major, “showstopper”, bugs.

Best regards

calle

On 12 July 2018 at 13:15, Morten Olav Hansen morten@dhis2.org wrote:

Ok, thanks Stian (no need to work on this then Viet)

(knut, using pg_dump we normally filter away analytic* which means no

period boundaries…)

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Thu, Jul 12, 2018 at 6:14 PM, Stian Sandvold stian@dhis2.org wrote:

I have started the work on renaming the table. I will update this thread

as soon as I make progress.

On Thu, Jul 12, 2018 at 12:17 PM, Morten Olav Hansen morten@dhis2.org

wrote:

Hi Knut

I mean the main issue the thread was about… using maintenance =>

clear analytic tables, it will delete analytics* which includes the

analytical boundaries.

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Thu, Jul 12, 2018 at 5:11 PM, Knut Staring knutst@gmail.com wrote:

Hi Morten, sorry but which functionality are you suggesting should not

be used? What do you mean by manually deleting?

Thanks,

Knut

On Thu, Jul 12, 2018, 4:23 PM Morten Olav Hansen morten@dhis2.org

wrote:

Hi Calle

We are aware of this issue (actually it caused a problem with us in

Lao), for now… I would also say don’t use this functionality, its better to

manually delete the analytics_* tables if you need it. We will rename that

table soon we hope, and that should fix it (this also causes potential

issues with your backups…)

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Thu, Jul 12, 2018 at 3:58 PM, Calle Hedberg

calle.hedberg@gmail.com wrote:

Bob,

No response/action on the JIRA bug report yet - I guess most

developers are on leave (wonderful summer here in Norway this year).

Otherwise I agree, the name of that table does not fit the general

naming convention as far as I can see. It would make more sense to call it

e.g. “programindicator_periodboundary”. The name then provides an intuitive

description of the content and it sorts together with the group of

programindicator tables.

Regards

calle

On 12 July 2018 at 08:57, Bob Jolliffe bobjolliffe@gmail.com

wrote:

Thats nasty alright. I guess using “-T anlytics_*” instead would

help. But there are so many backup scripts out there broken by

this

that it will be better to rename the table.

On 11 July 2018 at 22:39, Calle Hedberg calle.hedberg@gmail.com

wrote:

Hi

For as long as I can remember, we have used the standard

parameter "-T

analytics*" when dumping a DHIS2 database into e.g. a backup or

similar.

The purpose of the parameter was to exclude all analytics tables

from the

dump, since it is significantly faster to restore a dump without

analytics

tables and then run analytics to re-create them (due to the use

of

multi-threading), compared to dumping and restoring a database

instance with

all the analytics table (restore is NOT using multi-threading).

For some reason, in 2.29 a new table that stores periodboundary

data for

Program Indicators was called “analyticsperiodboundary” - which

means the

standard pg_dump parameter will leave that table behind together

with all

other “analytics*” tables.

Furthermore, the routine called “Clear Analytics Tables” found

under Data

Administration -> Maintenance is as before deleting all tables

named

Analytics* - THE PROBLEM IS THAT IT ALSO DELETES THE NEW

ANALYTICSPERIODBOUNDARY TABLE (same in both 2.29 and 2.30)

Which will crash your system in the sense that you won’t see any

program

indicator data in dashboards etc.

The “analyticsperiodboundary” table will be re-created and

re-populated with

DEFAULT (boundless) Program Indicator Period boundaries when you

re-start

the system (it’s part of the TableAlteror routine during

startup), but

  • you have to re-start the system
  • you will lose any non-default boundary settings used for any

program

indicator.

This has also been reported as a high-priority bug on JIRA

(DHIS2-4260).

Regards

Calle


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg



Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg



Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Stian Sandvold

Software developer, DHIS2

University of Oslo

http://www.dhis2.org


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg



Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Knut Staring

Department of Information, Evidence and Research

World Health Organization, Geneva, Switzerland

Office: +41 22 791 3683 Mob1: +33 6 4434 2931 Mob2: +47 9188 0522

Skype: knutstar


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Stian Sandvold
Software developer, DHIS2

University of Oslo

http://www.dhis2.org

Morten

Apology accepted - sometimes it’s tricky to handle those curve-balls. In any case, there are some lessons here…

:wink:

Regards

Calle

···

On 12 July 2018 at 13:51, Morten Olav Hansen morten@dhis2.org wrote:

Hi Calle

I have only been aware of this bug for a few days (when it hit us in Lao), and then we have both release and summer holidays going on. I agree the response is not ideal, but its not the easiest time for fixing bug issues.

We are working hard on fixing this now (it might take 1-2 days), and at that time we will recommend that everyone updates their 229 and 230 instances (I will leave that up to Stian when he is ready)

This was a design issue, and they are not that easy to fix (without causing havoc internally), but hopefully we have learned from this and we won’t name domain tables analytics* anymore.

Morten Olav Hansen

Senior Engineer, DHIS 2

Team Integration Lead

University of Oslo

http://www.dhis2.org

On Thu, Jul 12, 2018 at 6:33 PM, Calle Hedberg calle.hedberg@gmail.com wrote:

Hi

I’m glad it’s being addressed - but I am less happy to hear you are aware of it but haven’t said anything to the community… These type of bugs causes havoc, and waste a lot of time for users affected while they try to figure out what’s gone wrong.

There are two major lessons here:

Firstly, to stick to a standard and manageable naming convention for both tables and fields (and interfaces).

Secondly, to inform the user community promptly when you become aware of major, “showstopper”, bugs.

Best regards

calle

On 12 July 2018 at 13:15, Morten Olav Hansen morten@dhis2.org wrote:

Ok, thanks Stian (no need to work on this then Viet)

(knut, using pg_dump we normally filter away analytic* which means no period boundaries…)


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg


Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Thu, Jul 12, 2018 at 6:14 PM, Stian Sandvold stian@dhis2.org wrote:

I have started the work on renaming the table. I will update this thread as soon as I make progress.

On Thu, Jul 12, 2018 at 12:17 PM, Morten Olav Hansen morten@dhis2.org wrote:

Hi Knut

I mean the main issue the thread was about… using maintenance => clear analytic tables, it will delete analytics* which includes the analytical boundaries.


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Software developer, DHIS2

Stian Sandvold
University of Oslo

http://www.dhis2.org

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Thu, Jul 12, 2018 at 5:11 PM, Knut Staring knutst@gmail.com wrote:

Hi Morten, sorry but which functionality are you suggesting should not be used? What do you mean by manually deleting?

Thanks,

Knut

On Thu, Jul 12, 2018, 4:23 PM Morten Olav Hansen morten@dhis2.org wrote:

Hi Calle

We are aware of this issue (actually it caused a problem with us in Lao), for now… I would also say don’t use this functionality, its better to manually delete the analytics_* tables if you need it. We will rename that table soon we hope, and that should fix it (this also causes potential issues with your backups…)


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Thu, Jul 12, 2018 at 3:58 PM, Calle Hedberg calle.hedberg@gmail.com wrote:

Bob,

No response/action on the JIRA bug report yet - I guess most developers are on leave (wonderful summer here in Norway this year).

Otherwise I agree, the name of that table does not fit the general naming convention as far as I can see. It would make more sense to call it e.g. “programindicator_periodboundary”. The name then provides an intuitive description of the content and it sorts together with the group of programindicator tables.

Regards

calle


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

On 12 July 2018 at 08:57, Bob Jolliffe bobjolliffe@gmail.com wrote:

Thats nasty alright. I guess using “-T anlytics_*” instead would

help. But there are so many backup scripts out there broken by this

that it will be better to rename the table.

On 11 July 2018 at 22:39, Calle Hedberg calle.hedberg@gmail.com wrote:

Hi

For as long as I can remember, we have used the standard parameter "-T

analytics*" when dumping a DHIS2 database into e.g. a backup or similar.

The purpose of the parameter was to exclude all analytics tables from the

dump, since it is significantly faster to restore a dump without analytics

tables and then run analytics to re-create them (due to the use of

multi-threading), compared to dumping and restoring a database instance with

all the analytics table (restore is NOT using multi-threading).

For some reason, in 2.29 a new table that stores periodboundary data for

Program Indicators was called “analyticsperiodboundary” - which means the

standard pg_dump parameter will leave that table behind together with all

other “analytics*” tables.

Furthermore, the routine called “Clear Analytics Tables” found under Data

Administration -> Maintenance is as before deleting all tables named

Analytics* - THE PROBLEM IS THAT IT ALSO DELETES THE NEW

ANALYTICSPERIODBOUNDARY TABLE (same in both 2.29 and 2.30)

Which will crash your system in the sense that you won’t see any program

indicator data in dashboards etc.

The “analyticsperiodboundary” table will be re-created and re-populated with

DEFAULT (boundless) Program Indicator Period boundaries when you re-start

the system (it’s part of the TableAlteror routine during startup), but

  • you have to re-start the system
  • you will lose any non-default boundary settings used for any program

indicator.

This has also been reported as a high-priority bug on JIRA (DHIS2-4260).

Regards

Calle


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg



Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg



Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg


Apologies if this has been answered already, I haven’t been following closely, but has this issue been resolved and how was it resolved? Is it still OK to backup sans analytics tables?

Regards
Ed

1 Like

Edward,

yes, it was resolved over a year ago by renaming the “analyticsPeriodboundary” table. So as long as you are using a build after the fix (check for table “periodboundary”), then use backup sans analytics tables.

regards

Calle

2 Likes

Thank you for this information. You saved us from one big disaster.