Aggregating values

There is nothing being logged.

There are no views.

Caveman

···

On Wed, Jul 14, 2010 at 9:03 AM, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Caveman,

Can you please post a stack trace of your application (use

pastebin.com) from startup until you start the datamart/report table

process?

Do you have any external views that are linked to the report table you

are attempting to aggregate?

Regards,

Jason

On Wed, Jul 14, 2010 at 8:57 AM, Orvalho Augusto orvaquim@gmail.com wrote:

And even do those resource tables on Postgres it does not aggregate.

Caveman

On Wed, Jul 14, 2010 at 6:35 AM, Orvalho Augusto orvaquim@gmail.com wrote:

I have changed the database (from MySQL to Postgres)… and used DHIS

tools to export and import the data.

Well, those resource tables are being built in seconds on Postgres now. So

I think there is a bug with MySQL. And I want to help to correct it.

One verry bad consequence of not importing the dataelementids is: The

entering forms does not work no more. We have to redesign the forms after an

import.

Caveman

On Tue, Jul 13, 2010 at 10:28 PM, Orvalho Augusto orvaquim@gmail.com > > >> wrote:

Ok.

On Tue, Jul 13, 2010 at 11:55 AM, Bob Jolliffe bobjolliffe@gmail.com > > >>> wrote:

On 13 July 2010 07:17, Orvalho Augusto orvaquim@gmail.com wrote:

I recompiled DHIS2.05 yesterday with help of a magician Jason

yesterday. And

the first thing I tried was this aggregation thing.

So I started redoing the orgunitstructure. It gave the same error.

While it was running I used the

show full processlist

to find out that it is a verry long insert to be run at once. I have

1664

orgunits. So it sounds like indexing trouble.

As there is a faith and “antimysql fight” I am moving my data to

postgres. I

have to do it manually.

There is no mysql fight. Just more people using postgres I guess.

I have to do it manually because DHIS does not keep de ids fixed

(organisationunid, dataelementid etc). That affects our mapping data

for

data importation from other systems. So if someone could suggest a

good

strategy is welcomed.

How are you doing mapping data from other systems? If you tell me a

bit about these systems and the data they have maybe we can make a

plan :slight_smile:

Basically these systems are:

The dataset (the DHIS concept) is represented by a table (in EpiInfo

6.06, DBF or MSAccess) where each record should have a month (or week or

quarter), year, province, district and health center columns - sometimes

they place the name of the association or a special sector of the health

center - then the other columns define a dataelement for DHIS.

So what we were doing is:

  1. Migrate the data from anywhere (EpiInfo, MSAccess and DBF) to MySQL.
  1. On MySQL we have tables where the columns names of those tables

mapping to the dataelementid (which Lars recommend not be like this)

  1. There is a script on Python made to run routinely to process this data

and insert the new one on DHIS 2 tables.

When I say routinely, I mean each month or whenever I want it.

The objective were to use solely DHIS for report and one day for data

entry.

Caveman

Cheers

Bob

Thanks for your patience guys

Caveman

2010/7/12 Jason Pickering jason.p.pickering@gmail.com

Hi Lars,

Well, yes and no. If the resource tables (namely orgunitstructure) is

not generated, it would not be possible to assign any orgunits to a

given data mart/report table? Likewise, if the orgunit levels have

not

been defined, it is also not possible to assign any orgunits to a

given report table/datamart, or perhaps I am just hallucinating ? I

have run into this problem several times actually when creating new

databases. Ideally, the orgunitstructure table should be regenerated

when there have been changes to the orgunit hierarchy and users

should

be warned that it is not possible to create a report table/datamart

without first defining orgunit level names.

Regards,

Jason

2010/7/12 Lars Helge Øverland larshelge@gmail.com:

Generating any of the resource tables don’t affect the aggregation

/

report tables…

On Sun, Jul 11, 2010 at 5:36 PM, Orvalho Augusto > > >>>> >> > orvaquim@gmail.com > > >>>> >> > wrote:

Thanks!!!

Caveman

On Sun, Jul 11, 2010 at 5:03 PM, Jason Pickering > > >>>> >> >> jason.p.pickering@gmail.com wrote:

A very common mistake is not having generated the resource

tables,

namely “orgunitstructure”. Consult the documentation for more

details

on how to generate these tables.

Regards,

Jason

On Sun, Jul 11, 2010 at 4:27 PM, Orvalho Augusto > > >>>> >> >>> orvaquim@gmail.com > > >>>> >> >>> wrote:

I have a dataelement collected at a Health Centre level (eg:

ats_posfem)

in

one province and only in some particular districts. In

unpredictible

future

at National level we will be collecting on every health centre.

Now, my ignorance: How can I obtain the provincial total on

Report

Table? I

have been battled with DHIS so far and nothing.

Thanks guys

Caveman


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190

Hi Caveman,
There must be something being logged in apache-tomcat-X.X.XX/logs directory. Open up a terminal, and then do tail -f catalina.out and in a separate terminal, start Tomcat. Post the result and we may be able to help.

Let me know if something is not clear.

Regards,
Jason

···

On Wed, Jul 14, 2010 at 4:40 PM, Orvalho Augusto orvaquim@gmail.com wrote:

There is nothing being logged.

There are no views.

Caveman

On Wed, Jul 14, 2010 at 9:03 AM, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Caveman,

Can you please post a stack trace of your application (use

pastebin.com) from startup until you start the datamart/report table

process?

Do you have any external views that are linked to the report table you

are attempting to aggregate?

Regards,

Jason

On Wed, Jul 14, 2010 at 8:57 AM, Orvalho Augusto orvaquim@gmail.com wrote:

And even do those resource tables on Postgres it does not aggregate.

Caveman

On Wed, Jul 14, 2010 at 6:35 AM, Orvalho Augusto orvaquim@gmail.com wrote:

I have changed the database (from MySQL to Postgres)… and used DHIS

tools to export and import the data.

Well, those resource tables are being built in seconds on Postgres now. So

I think there is a bug with MySQL. And I want to help to correct it.

One verry bad consequence of not importing the dataelementids is: The

entering forms does not work no more. We have to redesign the forms after an

import.

Caveman

On Tue, Jul 13, 2010 at 10:28 PM, Orvalho Augusto orvaquim@gmail.com > > > > >> wrote:

Ok.

On Tue, Jul 13, 2010 at 11:55 AM, Bob Jolliffe bobjolliffe@gmail.com > > > > >>> wrote:

On 13 July 2010 07:17, Orvalho Augusto orvaquim@gmail.com wrote:

I recompiled DHIS2.05 yesterday with help of a magician Jason

yesterday. And

the first thing I tried was this aggregation thing.

So I started redoing the orgunitstructure. It gave the same error.

While it was running I used the

show full processlist

to find out that it is a verry long insert to be run at once. I have

1664

orgunits. So it sounds like indexing trouble.

As there is a faith and “antimysql fight” I am moving my data to

postgres. I

have to do it manually.

There is no mysql fight. Just more people using postgres I guess.

I have to do it manually because DHIS does not keep de ids fixed

(organisationunid, dataelementid etc). That affects our mapping data

for

data importation from other systems. So if someone could suggest a

good

strategy is welcomed.

How are you doing mapping data from other systems? If you tell me a

bit about these systems and the data they have maybe we can make a

plan :slight_smile:

Basically these systems are:

The dataset (the DHIS concept) is represented by a table (in EpiInfo

6.06, DBF or MSAccess) where each record should have a month (or week or

quarter), year, province, district and health center columns - sometimes

they place the name of the association or a special sector of the health

center - then the other columns define a dataelement for DHIS.

So what we were doing is:

  1. Migrate the data from anywhere (EpiInfo, MSAccess and DBF) to MySQL.
  1. On MySQL we have tables where the columns names of those tables

mapping to the dataelementid (which Lars recommend not be like this)

  1. There is a script on Python made to run routinely to process this data

and insert the new one on DHIS 2 tables.

When I say routinely, I mean each month or whenever I want it.

The objective were to use solely DHIS for report and one day for data

entry.

Caveman

Cheers

Bob

Thanks for your patience guys

Caveman

2010/7/12 Jason Pickering jason.p.pickering@gmail.com

Hi Lars,

Well, yes and no. If the resource tables (namely orgunitstructure) is

not generated, it would not be possible to assign any orgunits to a

given data mart/report table? Likewise, if the orgunit levels have

not

been defined, it is also not possible to assign any orgunits to a

given report table/datamart, or perhaps I am just hallucinating ? I

have run into this problem several times actually when creating new

databases. Ideally, the orgunitstructure table should be regenerated

when there have been changes to the orgunit hierarchy and users

should

be warned that it is not possible to create a report table/datamart

without first defining orgunit level names.

Regards,

Jason

2010/7/12 Lars Helge Øverland larshelge@gmail.com:

Generating any of the resource tables don’t affect the aggregation

/

report tables…

On Sun, Jul 11, 2010 at 5:36 PM, Orvalho Augusto > > > > >>>> >> > orvaquim@gmail.com > > > > >>>> >> > wrote:

Thanks!!!

Caveman

On Sun, Jul 11, 2010 at 5:03 PM, Jason Pickering > > > > >>>> >> >> jason.p.pickering@gmail.com wrote:

A very common mistake is not having generated the resource

tables,

namely “orgunitstructure”. Consult the documentation for more

details

on how to generate these tables.

Regards,

Jason

On Sun, Jul 11, 2010 at 4:27 PM, Orvalho Augusto > > > > >>>> >> >>> orvaquim@gmail.com > > > > >>>> >> >>> wrote:

I have a dataelement collected at a Health Centre level (eg:

ats_posfem)

in

one province and only in some particular districts. In

unpredictible

future

at National level we will be collecting on every health centre.

Now, my ignorance: How can I obtain the provincial total on

Report

Table? I

have been battled with DHIS so far and nothing.

Thanks guys

Caveman


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190

Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+260968395190

I am saying the truth.

This is what I have on my catalina.out:

  • INFO 16:31:50,284 User login success: ‘admin’ (DefaultUserAuditService.java [http-8081-3])
    Jul 14, 2010 4:31:50 PM org.apache.velocity.runtime.log.JdkLogChute log

SEVERE: #parse() null argument

Caveman

···

On Wed, Jul 14, 2010 at 5:35 PM, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Caveman,
There must be something being logged in apache-tomcat-X.X.XX/logs directory. Open up a terminal, and then do tail -f catalina.out and in a separate terminal, start Tomcat. Post the result and we may be able to help.

Let me know if something is not clear.

Regards,
Jason

On Wed, Jul 14, 2010 at 4:40 PM, Orvalho Augusto orvaquim@gmail.com wrote:

There is nothing being logged.

There are no views.

Caveman

On Wed, Jul 14, 2010 at 9:03 AM, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Caveman,

Can you please post a stack trace of your application (use

pastebin.com) from startup until you start the datamart/report table

process?

Do you have any external views that are linked to the report table you

are attempting to aggregate?

Regards,

Jason

On Wed, Jul 14, 2010 at 8:57 AM, Orvalho Augusto orvaquim@gmail.com wrote:

And even do those resource tables on Postgres it does not aggregate.

Caveman

On Wed, Jul 14, 2010 at 6:35 AM, Orvalho Augusto orvaquim@gmail.com wrote:

I have changed the database (from MySQL to Postgres)… and used DHIS

tools to export and import the data.

Well, those resource tables are being built in seconds on Postgres now. So

I think there is a bug with MySQL. And I want to help to correct it.

One verry bad consequence of not importing the dataelementids is: The

entering forms does not work no more. We have to redesign the forms after an

import.

Caveman

On Tue, Jul 13, 2010 at 10:28 PM, Orvalho Augusto orvaquim@gmail.com > > > > > > >> wrote:

Ok.

On Tue, Jul 13, 2010 at 11:55 AM, Bob Jolliffe bobjolliffe@gmail.com > > > > > > >>> wrote:

On 13 July 2010 07:17, Orvalho Augusto orvaquim@gmail.com wrote:

I recompiled DHIS2.05 yesterday with help of a magician Jason

yesterday. And

the first thing I tried was this aggregation thing.

So I started redoing the orgunitstructure. It gave the same error.

While it was running I used the

show full processlist

to find out that it is a verry long insert to be run at once. I have

1664

orgunits. So it sounds like indexing trouble.

As there is a faith and “antimysql fight” I am moving my data to

postgres. I

have to do it manually.

There is no mysql fight. Just more people using postgres I guess.

I have to do it manually because DHIS does not keep de ids fixed

(organisationunid, dataelementid etc). That affects our mapping data

for

data importation from other systems. So if someone could suggest a

good

strategy is welcomed.

How are you doing mapping data from other systems? If you tell me a

bit about these systems and the data they have maybe we can make a

plan :slight_smile:

Basically these systems are:

The dataset (the DHIS concept) is represented by a table (in EpiInfo

6.06, DBF or MSAccess) where each record should have a month (or week or

quarter), year, province, district and health center columns - sometimes

they place the name of the association or a special sector of the health

center - then the other columns define a dataelement for DHIS.

So what we were doing is:

  1. Migrate the data from anywhere (EpiInfo, MSAccess and DBF) to MySQL.
  1. On MySQL we have tables where the columns names of those tables

mapping to the dataelementid (which Lars recommend not be like this)

  1. There is a script on Python made to run routinely to process this data

and insert the new one on DHIS 2 tables.

When I say routinely, I mean each month or whenever I want it.

The objective were to use solely DHIS for report and one day for data

entry.

Caveman

Cheers

Bob

Thanks for your patience guys

Caveman

2010/7/12 Jason Pickering jason.p.pickering@gmail.com

Hi Lars,

Well, yes and no. If the resource tables (namely orgunitstructure) is

not generated, it would not be possible to assign any orgunits to a

given data mart/report table? Likewise, if the orgunit levels have

not

been defined, it is also not possible to assign any orgunits to a

given report table/datamart, or perhaps I am just hallucinating ? I

have run into this problem several times actually when creating new

databases. Ideally, the orgunitstructure table should be regenerated

when there have been changes to the orgunit hierarchy and users

should

be warned that it is not possible to create a report table/datamart

without first defining orgunit level names.

Regards,

Jason

2010/7/12 Lars Helge Øverland larshelge@gmail.com:

Generating any of the resource tables don’t affect the aggregation

/

report tables…

On Sun, Jul 11, 2010 at 5:36 PM, Orvalho Augusto > > > > > > >>>> >> > orvaquim@gmail.com > > > > > > >>>> >> > wrote:

Thanks!!!

Caveman

On Sun, Jul 11, 2010 at 5:03 PM, Jason Pickering > > > > > > >>>> >> >> jason.p.pickering@gmail.com wrote:

A very common mistake is not having generated the resource

tables,

namely “orgunitstructure”. Consult the documentation for more

details

on how to generate these tables.

Regards,

Jason

On Sun, Jul 11, 2010 at 4:27 PM, Orvalho Augusto > > > > > > >>>> >> >>> orvaquim@gmail.com > > > > > > >>>> >> >>> wrote:

I have a dataelement collected at a Health Centre level (eg:

ats_posfem)

in

one province and only in some particular districts. In

unpredictible

future

at National level we will be collecting on every health centre.

Now, my ignorance: How can I obtain the provincial total on

Report

Table? I

have been battled with DHIS so far and nothing.

Thanks guys

Caveman


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190


Jason P. Pickering
email: jason.p.pickering@gmail.com

tel:+260968395190

Hi Caveman,
Not doubting you but I have not seen DHIS start without some more information.
You see nothing similar to this when DHIS starts up?

  • INFO 11:15:58,359 Executing startup routine [1 of 13, runlevel 0]: DataSetSho

rtNamePopulator (DefaultStartupRoutineExecutor.java [main])

  • INFO 11:15:58,529 Populated DataSet short name and code (DataSetShortNamePopu
    lator.java [main])
  • INFO 11:15:58,529 Executing startup routine [2 of 13, runlevel 0]: UuidPopula

tor (DefaultStartupRoutineExecutor.java [main])

  • INFO 11:15:58,536 Checked CategoryOption uuids (UuidPopulator.java [main])

  • INFO 11:15:58,543 Checked Category uuids (UuidPopulator.java [main])

  • INFO 11:15:58,561 Checked DataElement uuids (UuidPopulator.java [main])

  • INFO 11:15:58,561 Executing startup routine [3 of 13, runlevel 0]: Organisati
    onUnitHierarchyVerifier (DefaultStartupRoutineExecutor.java [main])

When you replaced the DHIS war file, did you also remove the dhis directory (rm -rf dhis) ?

Regards,
Jason

···

On Wed, Jul 14, 2010 at 6:24 PM, Orvalho Augusto orvaquim@gmail.com wrote:

I am saying the truth.

This is what I have on my catalina.out:

  • INFO 16:31:50,284 User login success: ‘admin’ (DefaultUserAuditService.java [http-8081-3])
    Jul 14, 2010 4:31:50 PM org.apache.velocity.runtime.log.JdkLogChute log

SEVERE: #parse() null argument

Caveman

On Wed, Jul 14, 2010 at 5:35 PM, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Caveman,
There must be something being logged in apache-tomcat-X.X.XX/logs directory. Open up a terminal, and then do tail -f catalina.out and in a separate terminal, start Tomcat. Post the result and we may be able to help.

Let me know if something is not clear.

Regards,
Jason

On Wed, Jul 14, 2010 at 4:40 PM, Orvalho Augusto orvaquim@gmail.com wrote:

There is nothing being logged.

There are no views.

Caveman

On Wed, Jul 14, 2010 at 9:03 AM, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Caveman,

Can you please post a stack trace of your application (use

pastebin.com) from startup until you start the datamart/report table

process?

Do you have any external views that are linked to the report table you

are attempting to aggregate?

Regards,

Jason

On Wed, Jul 14, 2010 at 8:57 AM, Orvalho Augusto orvaquim@gmail.com wrote:

And even do those resource tables on Postgres it does not aggregate.

Caveman

On Wed, Jul 14, 2010 at 6:35 AM, Orvalho Augusto orvaquim@gmail.com wrote:

I have changed the database (from MySQL to Postgres)… and used DHIS

tools to export and import the data.

Well, those resource tables are being built in seconds on Postgres now. So

I think there is a bug with MySQL. And I want to help to correct it.

One verry bad consequence of not importing the dataelementids is: The

entering forms does not work no more. We have to redesign the forms after an

import.

Caveman

On Tue, Jul 13, 2010 at 10:28 PM, Orvalho Augusto orvaquim@gmail.com > > > > > > > > >> wrote:

Ok.

On Tue, Jul 13, 2010 at 11:55 AM, Bob Jolliffe bobjolliffe@gmail.com > > > > > > > > >>> wrote:

On 13 July 2010 07:17, Orvalho Augusto orvaquim@gmail.com wrote:

I recompiled DHIS2.05 yesterday with help of a magician Jason

yesterday. And

the first thing I tried was this aggregation thing.

So I started redoing the orgunitstructure. It gave the same error.

While it was running I used the

show full processlist

to find out that it is a verry long insert to be run at once. I have

1664

orgunits. So it sounds like indexing trouble.

As there is a faith and “antimysql fight” I am moving my data to

postgres. I

have to do it manually.

There is no mysql fight. Just more people using postgres I guess.

I have to do it manually because DHIS does not keep de ids fixed

(organisationunid, dataelementid etc). That affects our mapping data

for

data importation from other systems. So if someone could suggest a

good

strategy is welcomed.

How are you doing mapping data from other systems? If you tell me a

bit about these systems and the data they have maybe we can make a

plan :slight_smile:

Basically these systems are:

The dataset (the DHIS concept) is represented by a table (in EpiInfo

6.06, DBF or MSAccess) where each record should have a month (or week or

quarter), year, province, district and health center columns - sometimes

they place the name of the association or a special sector of the health

center - then the other columns define a dataelement for DHIS.

So what we were doing is:

  1. Migrate the data from anywhere (EpiInfo, MSAccess and DBF) to MySQL.
  1. On MySQL we have tables where the columns names of those tables

mapping to the dataelementid (which Lars recommend not be like this)

  1. There is a script on Python made to run routinely to process this data

and insert the new one on DHIS 2 tables.

When I say routinely, I mean each month or whenever I want it.

The objective were to use solely DHIS for report and one day for data

entry.

Caveman

Cheers

Bob

Thanks for your patience guys

Caveman

2010/7/12 Jason Pickering jason.p.pickering@gmail.com

Hi Lars,

Well, yes and no. If the resource tables (namely orgunitstructure) is

not generated, it would not be possible to assign any orgunits to a

given data mart/report table? Likewise, if the orgunit levels have

not

been defined, it is also not possible to assign any orgunits to a

given report table/datamart, or perhaps I am just hallucinating ? I

have run into this problem several times actually when creating new

databases. Ideally, the orgunitstructure table should be regenerated

when there have been changes to the orgunit hierarchy and users

should

be warned that it is not possible to create a report table/datamart

without first defining orgunit level names.

Regards,

Jason

2010/7/12 Lars Helge Øverland larshelge@gmail.com:

Generating any of the resource tables don’t affect the aggregation

/

report tables…

On Sun, Jul 11, 2010 at 5:36 PM, Orvalho Augusto > > > > > > > > >>>> >> > orvaquim@gmail.com > > > > > > > > >>>> >> > wrote:

Thanks!!!

Caveman

On Sun, Jul 11, 2010 at 5:03 PM, Jason Pickering > > > > > > > > >>>> >> >> jason.p.pickering@gmail.com wrote:

A very common mistake is not having generated the resource

tables,

namely “orgunitstructure”. Consult the documentation for more

details

on how to generate these tables.

Regards,

Jason

On Sun, Jul 11, 2010 at 4:27 PM, Orvalho Augusto > > > > > > > > >>>> >> >>> orvaquim@gmail.com > > > > > > > > >>>> >> >>> wrote:

I have a dataelement collected at a Health Centre level (eg:

ats_posfem)

in

one province and only in some particular districts. In

unpredictible

future

at National level we will be collecting on every health centre.

Now, my ignorance: How can I obtain the provincial total on

Report

Table? I

have been battled with DHIS so far and nothing.

Thanks guys

Caveman


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190


Jason P. Pickering
email: jason.p.pickering@gmail.com

tel:+260968395190

Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+260968395190

I understood that you want since I click on the create resource tables.

This is what since tomcat starts begin I have:

···

===========================================================
Jul 14, 2010 6:41:14 PM org.apache.catalina.core.AprLifecycleListener init

INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /home/jfclere/jakarta-tomcat-co
nnectors/jni/native/.libs
Jul 14, 2010 6:41:14 PM org.apache.coyote.http11.Http11Protocol init

INFO: Initializing Coyote HTTP/1.1 on http-6080
Jul 14, 2010 6:41:14 PM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 750 ms
Jul 14, 2010 6:41:14 PM org.apache.catalina.core.StandardService start

INFO: Starting service Catalina
Jul 14, 2010 6:41:14 PM org.apache.catalina.core.StandardEngine start
INFO: Starting Servlet Engine: Apache Tomcat/6.0.18
Jul 14, 2010 6:41:14 PM org.apache.catalina.startup.HostConfig deployWAR

INFO: Deploying web application archive dhis.war
Jul 14, 2010 6:41:16 PM org.apache.catalina.core.ApplicationContext log
INFO: Initializing Spring root WebApplicationContext

  • INFO 18:41:28,013 System property dhis2.home not set (DefaultLocationManager.java [main])

  • INFO 18:41:28,026 Environment variable DHIS2_HOME points to /home/tomcatjason (DefaultLocationManager.java [main])

orderByFragment(order-by)
sortSpecification({sort specification})
sortKeySpecification(sort key)

> sortKey($PlaceHolder$.stageinprogram)
< sortKey(null)

< sortKeySpecification(null)
< sortSpecification(null)
< orderByFragment(null)

orderByFragment(order-by)
sortSpecification({sort specification})

sortKeySpecification(sort key)
sortKey($PlaceHolder$.stageinprogram)
< sortKey(null)
< sortKeySpecification(null)
< sortSpecification(null)
< orderByFragment(null)

orderByFragment(order-by)
sortSpecification({sort specification})
sortKeySpecification(sort key)
sortKey($PlaceHolder$.sortorder)
< sortKey(null)
< sortKeySpecification(null)

< sortSpecification(null)
< orderByFragment(null)

  • INFO 18:41:59,621 Executing startup routine [1 of 11, runlevel 1]: TableAlteror (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:00,555 Tables updated (TableAlteror.java [main])

  • INFO 18:42:00,580 Executing startup routine [2 of 11, runlevel 2]: PeriodTypePopulator (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:00,651 Executing startup routine [3 of 11, runlevel 3]: TableCreator (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:00,718 Table aggregateddatavalue exists (TableCreator.java [main])

  • INFO 18:42:00,721 Table aggregatedindicatorvalue exists (TableCreator.java [main])

  • INFO 18:42:00,724 Index crosstab exists on table datavalue (TableCreator.java [main])

  • INFO 18:42:00,727 Table aggregateddatasetcompleteness exists (TableCreator.java [main])

  • INFO 18:42:00,730 Table datavaluearchive exists (TableCreator.java [main])

  • INFO 18:42:00,730 Executing startup routine [4 of 11, runlevel 3]: DataElementDefaultDimensionPopulator (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:03,355 Executing startup routine [5 of 11, runlevel 3]: MultiDimensionExpressionUpgrader (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:03,392 Executing startup routine [6 of 11, runlevel 5]: DataSetShortNamePopulator (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:03,428 Populated DataSet short name and code (DataSetShortNamePopulator.java [main])

  • INFO 18:42:03,429 Executing startup routine [7 of 11, runlevel 5]: UuidPopulator (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:03,434 Checked CategoryOption uuids (UuidPopulator.java [main])

  • INFO 18:42:03,439 Checked Category uuids (UuidPopulator.java [main])

  • INFO 18:42:04,887 Checked DataElement uuids (UuidPopulator.java [main])

  • INFO 18:42:04,888 Executing startup routine [8 of 11, runlevel 5]: OrganisationUnitGroupSetPopulator (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:04,898 Executing startup routine [9 of 11, runlevel 5]: DataEntryFormPopulator (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:04,899 Removing datasetid column from dataentryform table (DataEntryFormPopulator.java [main])

  • INFO 18:42:04,903 Executing startup routine [10 of 11, runlevel 6]: OpenHealthDataSourceWriter (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:04,904 Environment variable OPENHEALTH_HOME not set (OpenHealthDataSourceWriter.java [main])

  • INFO 18:42:04,904 Executing startup routine [11 of 11, runlevel 8]: I18nUpgrader (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:05,365 All startup routines done (DefaultStartupRoutineExecutor.java [main])
    Jul 14, 2010 6:42:13 PM org.apache.catalina.core.ApplicationContext log
    INFO: ContextListener: contextInitialized()
    Jul 14, 2010 6:42:13 PM org.apache.catalina.core.ApplicationContext log

INFO: SessionListener: contextInitialized()
Jul 14, 2010 6:42:13 PM org.apache.coyote.http11.Http11Protocol start
INFO: Starting Coyote HTTP/1.1 on http-6080
Jul 14, 2010 6:42:13 PM org.apache.jk.common.ChannelSocket init

INFO: Port busy 7009 java.net.BindException: Address already in use
Jul 14, 2010 6:42:13 PM org.apache.jk.common.ChannelSocket init
INFO: JK: ajp13 listening on /0.0.0.0:7010
Jul 14, 2010 6:42:13 PM org.apache.jk.server.JkMain start

INFO: Jk running ID=1 time=0/37 config=null
Jul 14, 2010 6:42:13 PM org.apache.catalina.startup.Catalina start
INFO: Server startup in 58789 ms

  • INFO 18:43:45,108 User login success: ‘admin’ (DefaultUserAuditService.java [http-6080-2])

  • INFO 18:56:39,905 User login success: ‘admin’ (DefaultUserAuditService.java [http-6080-3])
    ========================================================================

See guys I am starting to feel bad. I am noticing problems that none is reporting neither confirm. So I admit there is something wrong with me… should I search a medical help? :expressionless:

Caveman

On Wed, Jul 14, 2010 at 7:06 PM, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Caveman,
Not doubting you but I have not seen DHIS start without some more information.

You see nothing similar to this when DHIS starts up?

  • INFO 11:15:58,359 Executing startup routine [1 of 13, runlevel 0]: DataSetSho

rtNamePopulator (DefaultStartupRoutineExecutor.java [main])

  • INFO 11:15:58,529 Populated DataSet short name and code (DataSetShortNamePopu
    lator.java [main])
  • INFO 11:15:58,529 Executing startup routine [2 of 13, runlevel 0]: UuidPopula

tor (DefaultStartupRoutineExecutor.java [main])

  • INFO 11:15:58,536 Checked CategoryOption uuids (UuidPopulator.java [main])

  • INFO 11:15:58,543 Checked Category uuids (UuidPopulator.java [main])

  • INFO 11:15:58,561 Checked DataElement uuids (UuidPopulator.java [main])

  • INFO 11:15:58,561 Executing startup routine [3 of 13, runlevel 0]: Organisati
    onUnitHierarchyVerifier (DefaultStartupRoutineExecutor.java [main])

When you replaced the DHIS war file, did you also remove the dhis directory (rm -rf dhis) ?

Regards,
Jason

On Wed, Jul 14, 2010 at 6:24 PM, Orvalho Augusto orvaquim@gmail.com wrote:

I am saying the truth.

This is what I have on my catalina.out:

  • INFO 16:31:50,284 User login success: ‘admin’ (DefaultUserAuditService.java [http-8081-3])
    Jul 14, 2010 4:31:50 PM org.apache.velocity.runtime.log.JdkLogChute log

SEVERE: #parse() null argument

Caveman

On Wed, Jul 14, 2010 at 5:35 PM, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Caveman,
There must be something being logged in apache-tomcat-X.X.XX/logs directory. Open up a terminal, and then do tail -f catalina.out and in a separate terminal, start Tomcat. Post the result and we may be able to help.

Let me know if something is not clear.

Regards,
Jason

On Wed, Jul 14, 2010 at 4:40 PM, Orvalho Augusto orvaquim@gmail.com wrote:

There is nothing being logged.

There are no views.

Caveman

On Wed, Jul 14, 2010 at 9:03 AM, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Caveman,

Can you please post a stack trace of your application (use

pastebin.com) from startup until you start the datamart/report table

process?

Do you have any external views that are linked to the report table you

are attempting to aggregate?

Regards,

Jason

On Wed, Jul 14, 2010 at 8:57 AM, Orvalho Augusto orvaquim@gmail.com wrote:

And even do those resource tables on Postgres it does not aggregate.

Caveman

On Wed, Jul 14, 2010 at 6:35 AM, Orvalho Augusto orvaquim@gmail.com wrote:

I have changed the database (from MySQL to Postgres)… and used DHIS

tools to export and import the data.

Well, those resource tables are being built in seconds on Postgres now. So

I think there is a bug with MySQL. And I want to help to correct it.

One verry bad consequence of not importing the dataelementids is: The

entering forms does not work no more. We have to redesign the forms after an

import.

Caveman

On Tue, Jul 13, 2010 at 10:28 PM, Orvalho Augusto orvaquim@gmail.com > > > > > > > > > > >> wrote:

Ok.

On Tue, Jul 13, 2010 at 11:55 AM, Bob Jolliffe bobjolliffe@gmail.com > > > > > > > > > > >>> wrote:

On 13 July 2010 07:17, Orvalho Augusto orvaquim@gmail.com wrote:

I recompiled DHIS2.05 yesterday with help of a magician Jason

yesterday. And

the first thing I tried was this aggregation thing.

So I started redoing the orgunitstructure. It gave the same error.

While it was running I used the

show full processlist

to find out that it is a verry long insert to be run at once. I have

1664

orgunits. So it sounds like indexing trouble.

As there is a faith and “antimysql fight” I am moving my data to

postgres. I

have to do it manually.

There is no mysql fight. Just more people using postgres I guess.

I have to do it manually because DHIS does not keep de ids fixed

(organisationunid, dataelementid etc). That affects our mapping data

for

data importation from other systems. So if someone could suggest a

good

strategy is welcomed.

How are you doing mapping data from other systems? If you tell me a

bit about these systems and the data they have maybe we can make a

plan :slight_smile:

Basically these systems are:

The dataset (the DHIS concept) is represented by a table (in EpiInfo

6.06, DBF or MSAccess) where each record should have a month (or week or

quarter), year, province, district and health center columns - sometimes

they place the name of the association or a special sector of the health

center - then the other columns define a dataelement for DHIS.

So what we were doing is:

  1. Migrate the data from anywhere (EpiInfo, MSAccess and DBF) to MySQL.
  1. On MySQL we have tables where the columns names of those tables

mapping to the dataelementid (which Lars recommend not be like this)

  1. There is a script on Python made to run routinely to process this data

and insert the new one on DHIS 2 tables.

When I say routinely, I mean each month or whenever I want it.

The objective were to use solely DHIS for report and one day for data

entry.

Caveman

Cheers

Bob

Thanks for your patience guys

Caveman

2010/7/12 Jason Pickering jason.p.pickering@gmail.com

Hi Lars,

Well, yes and no. If the resource tables (namely orgunitstructure) is

not generated, it would not be possible to assign any orgunits to a

given data mart/report table? Likewise, if the orgunit levels have

not

been defined, it is also not possible to assign any orgunits to a

given report table/datamart, or perhaps I am just hallucinating ? I

have run into this problem several times actually when creating new

databases. Ideally, the orgunitstructure table should be regenerated

when there have been changes to the orgunit hierarchy and users

should

be warned that it is not possible to create a report table/datamart

without first defining orgunit level names.

Regards,

Jason

2010/7/12 Lars Helge Øverland larshelge@gmail.com:

Generating any of the resource tables don’t affect the aggregation

/

report tables…

On Sun, Jul 11, 2010 at 5:36 PM, Orvalho Augusto > > > > > > > > > > >>>> >> > orvaquim@gmail.com > > > > > > > > > > >>>> >> > wrote:

Thanks!!!

Caveman

On Sun, Jul 11, 2010 at 5:03 PM, Jason Pickering > > > > > > > > > > >>>> >> >> jason.p.pickering@gmail.com wrote:

A very common mistake is not having generated the resource

tables,

namely “orgunitstructure”. Consult the documentation for more

details

on how to generate these tables.

Regards,

Jason

On Sun, Jul 11, 2010 at 4:27 PM, Orvalho Augusto > > > > > > > > > > >>>> >> >>> orvaquim@gmail.com > > > > > > > > > > >>>> >> >>> wrote:

I have a dataelement collected at a Health Centre level (eg:

ats_posfem)

in

one province and only in some particular districts. In

unpredictible

future

at National level we will be collecting on every health centre.

Now, my ignorance: How can I obtain the provincial total on

Report

Table? I

have been battled with DHIS so far and nothing.

Thanks guys

Caveman


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190


Jason P. Pickering
email: jason.p.pickering@gmail.com

tel:+260968395190


Jason P. Pickering
email: jason.p.pickering@gmail.com

tel:+260968395190

No need for medical help. I have had many of these sorts of problems as well, so you are not alone.

Can you post the log when you begin the aggregation process?

Regards,
Jason

···

On Wed, Jul 14, 2010 at 7:40 PM, Orvalho Augusto orvaquim@gmail.com wrote:

I understood that you want since I click on the create resource tables.

This is what since tomcat starts begin I have:

Jul 14, 2010 6:41:14 PM org.apache.catalina.core.AprLifecycleListener init

INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /home/jfclere/jakarta-tomcat-co
nnectors/jni/native/.libs
Jul 14, 2010 6:41:14 PM org.apache.coyote.http11.Http11Protocol init

INFO: Initializing Coyote HTTP/1.1 on http-6080
Jul 14, 2010 6:41:14 PM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 750 ms
Jul 14, 2010 6:41:14 PM org.apache.catalina.core.StandardService start

INFO: Starting service Catalina
Jul 14, 2010 6:41:14 PM org.apache.catalina.core.StandardEngine start
INFO: Starting Servlet Engine: Apache Tomcat/6.0.18
Jul 14, 2010 6:41:14 PM org.apache.catalina.startup.HostConfig deployWAR

INFO: Deploying web application archive dhis.war
Jul 14, 2010 6:41:16 PM org.apache.catalina.core.ApplicationContext log
INFO: Initializing Spring root WebApplicationContext

  • INFO 18:41:28,013 System property dhis2.home not set (DefaultLocationManager.java [main])

  • INFO 18:41:28,026 Environment variable DHIS2_HOME points to /home/tomcatjason (DefaultLocationManager.java [main])

orderByFragment(order-by)
sortSpecification({sort specification})
sortKeySpecification(sort key)

> sortKey($PlaceHolder$.stageinprogram)
< sortKey(null)

< sortKeySpecification(null)
< sortSpecification(null)
< orderByFragment(null)

orderByFragment(order-by)
sortSpecification({sort specification})

sortKeySpecification(sort key)
sortKey($PlaceHolder$.stageinprogram)
< sortKey(null)
< sortKeySpecification(null)
< sortSpecification(null)
< orderByFragment(null)

orderByFragment(order-by)
sortSpecification({sort specification})
sortKeySpecification(sort key)
sortKey($PlaceHolder$.sortorder)
< sortKey(null)
< sortKeySpecification(null)

< sortSpecification(null)
< orderByFragment(null)

  • INFO 18:41:59,621 Executing startup routine [1 of 11, runlevel 1]: TableAlteror (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:00,555 Tables updated (TableAlteror.java [main])

  • INFO 18:42:00,580 Executing startup routine [2 of 11, runlevel 2]: PeriodTypePopulator (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:00,651 Executing startup routine [3 of 11, runlevel 3]: TableCreator (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:00,718 Table aggregateddatavalue exists (TableCreator.java [main])

  • INFO 18:42:00,721 Table aggregatedindicatorvalue exists (TableCreator.java [main])

  • INFO 18:42:00,724 Index crosstab exists on table datavalue (TableCreator.java [main])

  • INFO 18:42:00,727 Table aggregateddatasetcompleteness exists (TableCreator.java [main])

  • INFO 18:42:00,730 Table datavaluearchive exists (TableCreator.java [main])

  • INFO 18:42:00,730 Executing startup routine [4 of 11, runlevel 3]: DataElementDefaultDimensionPopulator (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:03,355 Executing startup routine [5 of 11, runlevel 3]: MultiDimensionExpressionUpgrader (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:03,392 Executing startup routine [6 of 11, runlevel 5]: DataSetShortNamePopulator (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:03,428 Populated DataSet short name and code (DataSetShortNamePopulator.java [main])

  • INFO 18:42:03,429 Executing startup routine [7 of 11, runlevel 5]: UuidPopulator (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:03,434 Checked CategoryOption uuids (UuidPopulator.java [main])

  • INFO 18:42:03,439 Checked Category uuids (UuidPopulator.java [main])

  • INFO 18:42:04,887 Checked DataElement uuids (UuidPopulator.java [main])

  • INFO 18:42:04,888 Executing startup routine [8 of 11, runlevel 5]: OrganisationUnitGroupSetPopulator (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:04,898 Executing startup routine [9 of 11, runlevel 5]: DataEntryFormPopulator (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:04,899 Removing datasetid column from dataentryform table (DataEntryFormPopulator.java [main])

  • INFO 18:42:04,903 Executing startup routine [10 of 11, runlevel 6]: OpenHealthDataSourceWriter (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:04,904 Environment variable OPENHEALTH_HOME not set (OpenHealthDataSourceWriter.java [main])

  • INFO 18:42:04,904 Executing startup routine [11 of 11, runlevel 8]: I18nUpgrader (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:05,365 All startup routines done (DefaultStartupRoutineExecutor.java [main])
    Jul 14, 2010 6:42:13 PM org.apache.catalina.core.ApplicationContext log
    INFO: ContextListener: contextInitialized()
    Jul 14, 2010 6:42:13 PM org.apache.catalina.core.ApplicationContext log

INFO: SessionListener: contextInitialized()
Jul 14, 2010 6:42:13 PM org.apache.coyote.http11.Http11Protocol start
INFO: Starting Coyote HTTP/1.1 on http-6080
Jul 14, 2010 6:42:13 PM org.apache.jk.common.ChannelSocket init

INFO: Port busy 7009 java.net.BindException: Address already in use
Jul 14, 2010 6:42:13 PM org.apache.jk.common.ChannelSocket init
INFO: JK: ajp13 listening on /0.0.0.0:7010

Jul 14, 2010 6:42:13 PM org.apache.jk.server.JkMain start

INFO: Jk running ID=1 time=0/37 config=null
Jul 14, 2010 6:42:13 PM org.apache.catalina.startup.Catalina start
INFO: Server startup in 58789 ms

  • INFO 18:43:45,108 User login success: ‘admin’ (DefaultUserAuditService.java [http-6080-2])

  • INFO 18:56:39,905 User login success: ‘admin’ (DefaultUserAuditService.java [http-6080-3])
    ========================================================================

See guys I am starting to feel bad. I am noticing problems that none is reporting neither confirm. So I admit there is something wrong with me… should I search a medical help? :expressionless:

Caveman

On Wed, Jul 14, 2010 at 7:06 PM, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Caveman,
Not doubting you but I have not seen DHIS start without some more information.

You see nothing similar to this when DHIS starts up?

  • INFO 11:15:58,359 Executing startup routine [1 of 13, runlevel 0]: DataSetSho

rtNamePopulator (DefaultStartupRoutineExecutor.java [main])

  • INFO 11:15:58,529 Populated DataSet short name and code (DataSetShortNamePopu
    lator.java [main])
  • INFO 11:15:58,529 Executing startup routine [2 of 13, runlevel 0]: UuidPopula

tor (DefaultStartupRoutineExecutor.java [main])

  • INFO 11:15:58,536 Checked CategoryOption uuids (UuidPopulator.java [main])

  • INFO 11:15:58,543 Checked Category uuids (UuidPopulator.java [main])

  • INFO 11:15:58,561 Checked DataElement uuids (UuidPopulator.java [main])

  • INFO 11:15:58,561 Executing startup routine [3 of 13, runlevel 0]: Organisati
    onUnitHierarchyVerifier (DefaultStartupRoutineExecutor.java [main])

When you replaced the DHIS war file, did you also remove the dhis directory (rm -rf dhis) ?

Regards,
Jason

On Wed, Jul 14, 2010 at 6:24 PM, Orvalho Augusto orvaquim@gmail.com wrote:

I am saying the truth.

This is what I have on my catalina.out:

  • INFO 16:31:50,284 User login success: ‘admin’ (DefaultUserAuditService.java [http-8081-3])
    Jul 14, 2010 4:31:50 PM org.apache.velocity.runtime.log.JdkLogChute log

SEVERE: #parse() null argument

Caveman

On Wed, Jul 14, 2010 at 5:35 PM, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Caveman,
There must be something being logged in apache-tomcat-X.X.XX/logs directory. Open up a terminal, and then do tail -f catalina.out and in a separate terminal, start Tomcat. Post the result and we may be able to help.

Let me know if something is not clear.

Regards,
Jason

On Wed, Jul 14, 2010 at 4:40 PM, Orvalho Augusto orvaquim@gmail.com wrote:

There is nothing being logged.

There are no views.

Caveman

On Wed, Jul 14, 2010 at 9:03 AM, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Caveman,

Can you please post a stack trace of your application (use

pastebin.com) from startup until you start the datamart/report table

process?

Do you have any external views that are linked to the report table you

are attempting to aggregate?

Regards,

Jason

On Wed, Jul 14, 2010 at 8:57 AM, Orvalho Augusto orvaquim@gmail.com wrote:

And even do those resource tables on Postgres it does not aggregate.

Caveman

On Wed, Jul 14, 2010 at 6:35 AM, Orvalho Augusto orvaquim@gmail.com wrote:

I have changed the database (from MySQL to Postgres)… and used DHIS

tools to export and import the data.

Well, those resource tables are being built in seconds on Postgres now. So

I think there is a bug with MySQL. And I want to help to correct it.

One verry bad consequence of not importing the dataelementids is: The

entering forms does not work no more. We have to redesign the forms after an

import.

Caveman

On Tue, Jul 13, 2010 at 10:28 PM, Orvalho Augusto orvaquim@gmail.com > > > > > > > > > > > > >> wrote:

Ok.

On Tue, Jul 13, 2010 at 11:55 AM, Bob Jolliffe bobjolliffe@gmail.com > > > > > > > > > > > > >>> wrote:

On 13 July 2010 07:17, Orvalho Augusto orvaquim@gmail.com wrote:

I recompiled DHIS2.05 yesterday with help of a magician Jason

yesterday. And

the first thing I tried was this aggregation thing.

So I started redoing the orgunitstructure. It gave the same error.

While it was running I used the

show full processlist

to find out that it is a verry long insert to be run at once. I have

1664

orgunits. So it sounds like indexing trouble.

As there is a faith and “antimysql fight” I am moving my data to

postgres. I

have to do it manually.

There is no mysql fight. Just more people using postgres I guess.

I have to do it manually because DHIS does not keep de ids fixed

(organisationunid, dataelementid etc). That affects our mapping data

for

data importation from other systems. So if someone could suggest a

good

strategy is welcomed.

How are you doing mapping data from other systems? If you tell me a

bit about these systems and the data they have maybe we can make a

plan :slight_smile:

Basically these systems are:

The dataset (the DHIS concept) is represented by a table (in EpiInfo

6.06, DBF or MSAccess) where each record should have a month (or week or

quarter), year, province, district and health center columns - sometimes

they place the name of the association or a special sector of the health

center - then the other columns define a dataelement for DHIS.

So what we were doing is:

  1. Migrate the data from anywhere (EpiInfo, MSAccess and DBF) to MySQL.
  1. On MySQL we have tables where the columns names of those tables

mapping to the dataelementid (which Lars recommend not be like this)

  1. There is a script on Python made to run routinely to process this data

and insert the new one on DHIS 2 tables.

When I say routinely, I mean each month or whenever I want it.

The objective were to use solely DHIS for report and one day for data

entry.

Caveman

Cheers

Bob

Thanks for your patience guys

Caveman

2010/7/12 Jason Pickering jason.p.pickering@gmail.com

Hi Lars,

Well, yes and no. If the resource tables (namely orgunitstructure) is

not generated, it would not be possible to assign any orgunits to a

given data mart/report table? Likewise, if the orgunit levels have

not

been defined, it is also not possible to assign any orgunits to a

given report table/datamart, or perhaps I am just hallucinating ? I

have run into this problem several times actually when creating new

databases. Ideally, the orgunitstructure table should be regenerated

when there have been changes to the orgunit hierarchy and users

should

be warned that it is not possible to create a report table/datamart

without first defining orgunit level names.

Regards,

Jason

2010/7/12 Lars Helge Øverland larshelge@gmail.com:

Generating any of the resource tables don’t affect the aggregation

/

report tables…

On Sun, Jul 11, 2010 at 5:36 PM, Orvalho Augusto > > > > > > > > > > > > >>>> >> > orvaquim@gmail.com > > > > > > > > > > > > >>>> >> > wrote:

Thanks!!!

Caveman

On Sun, Jul 11, 2010 at 5:03 PM, Jason Pickering > > > > > > > > > > > > >>>> >> >> jason.p.pickering@gmail.com wrote:

A very common mistake is not having generated the resource

tables,

namely “orgunitstructure”. Consult the documentation for more

details

on how to generate these tables.

Regards,

Jason

On Sun, Jul 11, 2010 at 4:27 PM, Orvalho Augusto > > > > > > > > > > > > >>>> >> >>> orvaquim@gmail.com > > > > > > > > > > > > >>>> >> >>> wrote:

I have a dataelement collected at a Health Centre level (eg:

ats_posfem)

in

one province and only in some particular districts. In

unpredictible

future

at National level we will be collecting on every health centre.

Now, my ignorance: How can I obtain the provincial total on

Report

Table? I

have been battled with DHIS so far and nothing.

Thanks guys

Caveman


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190


Jason P. Pickering
email: jason.p.pickering@gmail.com

tel:+260968395190


Jason P. Pickering
email: jason.p.pickering@gmail.com

tel:+260968395190

Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+260968395190

Is all I have on the logs.

The thing it never finishs to run the creation of the organisation structure. And when I check MySQL is occupided running a very long INSERT query. Last time I left it from 12 hours and nothing happened.

Thanks

Caveman

···

On Wed, Jul 14, 2010 at 8:03 PM, Jason Pickering jason.p.pickering@gmail.com wrote:

No need for medical help. I have had many of these sorts of problems as well, so you are not alone.

Can you post the log when you begin the aggregation process?

Regards,
Jason

On Wed, Jul 14, 2010 at 7:40 PM, Orvalho Augusto orvaquim@gmail.com wrote:

I understood that you want since I click on the create resource tables.

This is what since tomcat starts begin I have:

Jul 14, 2010 6:41:14 PM org.apache.catalina.core.AprLifecycleListener init

INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /home/jfclere/jakarta-tomcat-co
nnectors/jni/native/.libs
Jul 14, 2010 6:41:14 PM org.apache.coyote.http11.Http11Protocol init

INFO: Initializing Coyote HTTP/1.1 on http-6080
Jul 14, 2010 6:41:14 PM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 750 ms
Jul 14, 2010 6:41:14 PM org.apache.catalina.core.StandardService start

INFO: Starting service Catalina
Jul 14, 2010 6:41:14 PM org.apache.catalina.core.StandardEngine start
INFO: Starting Servlet Engine: Apache Tomcat/6.0.18
Jul 14, 2010 6:41:14 PM org.apache.catalina.startup.HostConfig deployWAR

INFO: Deploying web application archive dhis.war
Jul 14, 2010 6:41:16 PM org.apache.catalina.core.ApplicationContext log
INFO: Initializing Spring root WebApplicationContext

  • INFO 18:41:28,013 System property dhis2.home not set (DefaultLocationManager.java [main])

  • INFO 18:41:28,026 Environment variable DHIS2_HOME points to /home/tomcatjason (DefaultLocationManager.java [main])

orderByFragment(order-by)
sortSpecification({sort specification})
sortKeySpecification(sort key)

> sortKey($PlaceHolder$.stageinprogram)
< sortKey(null)

< sortKeySpecification(null)
< sortSpecification(null)
< orderByFragment(null)

orderByFragment(order-by)
sortSpecification({sort specification})

sortKeySpecification(sort key)
sortKey($PlaceHolder$.stageinprogram)
< sortKey(null)
< sortKeySpecification(null)
< sortSpecification(null)
< orderByFragment(null)

orderByFragment(order-by)
sortSpecification({sort specification})
sortKeySpecification(sort key)
sortKey($PlaceHolder$.sortorder)
< sortKey(null)
< sortKeySpecification(null)

< sortSpecification(null)
< orderByFragment(null)

  • INFO 18:41:59,621 Executing startup routine [1 of 11, runlevel 1]: TableAlteror (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:00,555 Tables updated (TableAlteror.java [main])

  • INFO 18:42:00,580 Executing startup routine [2 of 11, runlevel 2]: PeriodTypePopulator (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:00,651 Executing startup routine [3 of 11, runlevel 3]: TableCreator (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:00,718 Table aggregateddatavalue exists (TableCreator.java [main])

  • INFO 18:42:00,721 Table aggregatedindicatorvalue exists (TableCreator.java [main])

  • INFO 18:42:00,724 Index crosstab exists on table datavalue (TableCreator.java [main])

  • INFO 18:42:00,727 Table aggregateddatasetcompleteness exists (TableCreator.java [main])

  • INFO 18:42:00,730 Table datavaluearchive exists (TableCreator.java [main])

  • INFO 18:42:00,730 Executing startup routine [4 of 11, runlevel 3]: DataElementDefaultDimensionPopulator (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:03,355 Executing startup routine [5 of 11, runlevel 3]: MultiDimensionExpressionUpgrader (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:03,392 Executing startup routine [6 of 11, runlevel 5]: DataSetShortNamePopulator (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:03,428 Populated DataSet short name and code (DataSetShortNamePopulator.java [main])

  • INFO 18:42:03,429 Executing startup routine [7 of 11, runlevel 5]: UuidPopulator (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:03,434 Checked CategoryOption uuids (UuidPopulator.java [main])

  • INFO 18:42:03,439 Checked Category uuids (UuidPopulator.java [main])

  • INFO 18:42:04,887 Checked DataElement uuids (UuidPopulator.java [main])

  • INFO 18:42:04,888 Executing startup routine [8 of 11, runlevel 5]: OrganisationUnitGroupSetPopulator (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:04,898 Executing startup routine [9 of 11, runlevel 5]: DataEntryFormPopulator (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:04,899 Removing datasetid column from dataentryform table (DataEntryFormPopulator.java [main])

  • INFO 18:42:04,903 Executing startup routine [10 of 11, runlevel 6]: OpenHealthDataSourceWriter (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:04,904 Environment variable OPENHEALTH_HOME not set (OpenHealthDataSourceWriter.java [main])

  • INFO 18:42:04,904 Executing startup routine [11 of 11, runlevel 8]: I18nUpgrader (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:05,365 All startup routines done (DefaultStartupRoutineExecutor.java [main])
    Jul 14, 2010 6:42:13 PM org.apache.catalina.core.ApplicationContext log
    INFO: ContextListener: contextInitialized()
    Jul 14, 2010 6:42:13 PM org.apache.catalina.core.ApplicationContext log

INFO: SessionListener: contextInitialized()
Jul 14, 2010 6:42:13 PM org.apache.coyote.http11.Http11Protocol start
INFO: Starting Coyote HTTP/1.1 on http-6080
Jul 14, 2010 6:42:13 PM org.apache.jk.common.ChannelSocket init

INFO: Port busy 7009 java.net.BindException: Address already in use
Jul 14, 2010 6:42:13 PM org.apache.jk.common.ChannelSocket init
INFO: JK: ajp13 listening on /0.0.0.0:7010

Jul 14, 2010 6:42:13 PM org.apache.jk.server.JkMain start

INFO: Jk running ID=1 time=0/37 config=null
Jul 14, 2010 6:42:13 PM org.apache.catalina.startup.Catalina start
INFO: Server startup in 58789 ms

  • INFO 18:43:45,108 User login success: ‘admin’ (DefaultUserAuditService.java [http-6080-2])

  • INFO 18:56:39,905 User login success: ‘admin’ (DefaultUserAuditService.java [http-6080-3])
    ========================================================================

See guys I am starting to feel bad. I am noticing problems that none is reporting neither confirm. So I admit there is something wrong with me… should I search a medical help? :expressionless:

Caveman

On Wed, Jul 14, 2010 at 7:06 PM, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Caveman,
Not doubting you but I have not seen DHIS start without some more information.

You see nothing similar to this when DHIS starts up?

  • INFO 11:15:58,359 Executing startup routine [1 of 13, runlevel 0]: DataSetSho

rtNamePopulator (DefaultStartupRoutineExecutor.java [main])

  • INFO 11:15:58,529 Populated DataSet short name and code (DataSetShortNamePopu
    lator.java [main])
  • INFO 11:15:58,529 Executing startup routine [2 of 13, runlevel 0]: UuidPopula

tor (DefaultStartupRoutineExecutor.java [main])

  • INFO 11:15:58,536 Checked CategoryOption uuids (UuidPopulator.java [main])

  • INFO 11:15:58,543 Checked Category uuids (UuidPopulator.java [main])

  • INFO 11:15:58,561 Checked DataElement uuids (UuidPopulator.java [main])

  • INFO 11:15:58,561 Executing startup routine [3 of 13, runlevel 0]: Organisati
    onUnitHierarchyVerifier (DefaultStartupRoutineExecutor.java [main])

When you replaced the DHIS war file, did you also remove the dhis directory (rm -rf dhis) ?

Regards,
Jason

On Wed, Jul 14, 2010 at 6:24 PM, Orvalho Augusto orvaquim@gmail.com wrote:

I am saying the truth.

This is what I have on my catalina.out:

  • INFO 16:31:50,284 User login success: ‘admin’ (DefaultUserAuditService.java [http-8081-3])
    Jul 14, 2010 4:31:50 PM org.apache.velocity.runtime.log.JdkLogChute log

SEVERE: #parse() null argument

Caveman

On Wed, Jul 14, 2010 at 5:35 PM, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Caveman,
There must be something being logged in apache-tomcat-X.X.XX/logs directory. Open up a terminal, and then do tail -f catalina.out and in a separate terminal, start Tomcat. Post the result and we may be able to help.

Let me know if something is not clear.

Regards,
Jason

On Wed, Jul 14, 2010 at 4:40 PM, Orvalho Augusto orvaquim@gmail.com wrote:

There is nothing being logged.

There are no views.

Caveman

On Wed, Jul 14, 2010 at 9:03 AM, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Caveman,

Can you please post a stack trace of your application (use

pastebin.com) from startup until you start the datamart/report table

process?

Do you have any external views that are linked to the report table you

are attempting to aggregate?

Regards,

Jason

On Wed, Jul 14, 2010 at 8:57 AM, Orvalho Augusto orvaquim@gmail.com wrote:

And even do those resource tables on Postgres it does not aggregate.

Caveman

On Wed, Jul 14, 2010 at 6:35 AM, Orvalho Augusto orvaquim@gmail.com wrote:

I have changed the database (from MySQL to Postgres)… and used DHIS

tools to export and import the data.

Well, those resource tables are being built in seconds on Postgres now. So

I think there is a bug with MySQL. And I want to help to correct it.

One verry bad consequence of not importing the dataelementids is: The

entering forms does not work no more. We have to redesign the forms after an

import.

Caveman

On Tue, Jul 13, 2010 at 10:28 PM, Orvalho Augusto orvaquim@gmail.com > > > > > > > > > > > > > > >> wrote:

Ok.

On Tue, Jul 13, 2010 at 11:55 AM, Bob Jolliffe bobjolliffe@gmail.com > > > > > > > > > > > > > > >>> wrote:

On 13 July 2010 07:17, Orvalho Augusto orvaquim@gmail.com wrote:

I recompiled DHIS2.05 yesterday with help of a magician Jason

yesterday. And

the first thing I tried was this aggregation thing.

So I started redoing the orgunitstructure. It gave the same error.

While it was running I used the

show full processlist

to find out that it is a verry long insert to be run at once. I have

1664

orgunits. So it sounds like indexing trouble.

As there is a faith and “antimysql fight” I am moving my data to

postgres. I

have to do it manually.

There is no mysql fight. Just more people using postgres I guess.

I have to do it manually because DHIS does not keep de ids fixed

(organisationunid, dataelementid etc). That affects our mapping data

for

data importation from other systems. So if someone could suggest a

good

strategy is welcomed.

How are you doing mapping data from other systems? If you tell me a

bit about these systems and the data they have maybe we can make a

plan :slight_smile:

Basically these systems are:

The dataset (the DHIS concept) is represented by a table (in EpiInfo

6.06, DBF or MSAccess) where each record should have a month (or week or

quarter), year, province, district and health center columns - sometimes

they place the name of the association or a special sector of the health

center - then the other columns define a dataelement for DHIS.

So what we were doing is:

  1. Migrate the data from anywhere (EpiInfo, MSAccess and DBF) to MySQL.
  1. On MySQL we have tables where the columns names of those tables

mapping to the dataelementid (which Lars recommend not be like this)

  1. There is a script on Python made to run routinely to process this data

and insert the new one on DHIS 2 tables.

When I say routinely, I mean each month or whenever I want it.

The objective were to use solely DHIS for report and one day for data

entry.

Caveman

Cheers

Bob

Thanks for your patience guys

Caveman

2010/7/12 Jason Pickering jason.p.pickering@gmail.com

Hi Lars,

Well, yes and no. If the resource tables (namely orgunitstructure) is

not generated, it would not be possible to assign any orgunits to a

given data mart/report table? Likewise, if the orgunit levels have

not

been defined, it is also not possible to assign any orgunits to a

given report table/datamart, or perhaps I am just hallucinating ? I

have run into this problem several times actually when creating new

databases. Ideally, the orgunitstructure table should be regenerated

when there have been changes to the orgunit hierarchy and users

should

be warned that it is not possible to create a report table/datamart

without first defining orgunit level names.

Regards,

Jason

2010/7/12 Lars Helge Øverland larshelge@gmail.com:

Generating any of the resource tables don’t affect the aggregation

/

report tables…

On Sun, Jul 11, 2010 at 5:36 PM, Orvalho Augusto > > > > > > > > > > > > > > >>>> >> > orvaquim@gmail.com > > > > > > > > > > > > > > >>>> >> > wrote:

Thanks!!!

Caveman

On Sun, Jul 11, 2010 at 5:03 PM, Jason Pickering > > > > > > > > > > > > > > >>>> >> >> jason.p.pickering@gmail.com wrote:

A very common mistake is not having generated the resource

tables,

namely “orgunitstructure”. Consult the documentation for more

details

on how to generate these tables.

Regards,

Jason

On Sun, Jul 11, 2010 at 4:27 PM, Orvalho Augusto > > > > > > > > > > > > > > >>>> >> >>> orvaquim@gmail.com > > > > > > > > > > > > > > >>>> >> >>> wrote:

I have a dataelement collected at a Health Centre level (eg:

ats_posfem)

in

one province and only in some particular districts. In

unpredictible

future

at National level we will be collecting on every health centre.

Now, my ignorance: How can I obtain the provincial total on

Report

Table? I

have been battled with DHIS so far and nothing.

Thanks guys

Caveman


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190


Jason P. Pickering
email: jason.p.pickering@gmail.com

tel:+260968395190


Jason P. Pickering
email: jason.p.pickering@gmail.com

tel:+260968395190


Jason P. Pickering
email: jason.p.pickering@gmail.com

tel:+260968395190

Without the logs when aggregation takes place, it will be difficult to determine what is happening.

Be sure that there are no sourceids in the “source” table without corresponding organisationanunitids in the organisationunit table.

This has caused problems for yours truly in the past. Trawl the mailing list for more information on this.

Regards,
JPP

···

On Wed, Jul 14, 2010 at 8:43 PM, Orvalho Augusto orvaquim@gmail.com wrote:

Is all I have on the logs.

The thing it never finishs to run the creation of the organisation structure. And when I check MySQL is occupided running a very long INSERT query. Last time I left it from 12 hours and nothing happened.

Thanks

Caveman

On Wed, Jul 14, 2010 at 8:03 PM, Jason Pickering jason.p.pickering@gmail.com wrote:

No need for medical help. I have had many of these sorts of problems as well, so you are not alone.

Can you post the log when you begin the aggregation process?

Regards,
Jason

On Wed, Jul 14, 2010 at 7:40 PM, Orvalho Augusto orvaquim@gmail.com wrote:

I understood that you want since I click on the create resource tables.

This is what since tomcat starts begin I have:

Jul 14, 2010 6:41:14 PM org.apache.catalina.core.AprLifecycleListener init

INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /home/jfclere/jakarta-tomcat-co
nnectors/jni/native/.libs
Jul 14, 2010 6:41:14 PM org.apache.coyote.http11.Http11Protocol init

INFO: Initializing Coyote HTTP/1.1 on http-6080
Jul 14, 2010 6:41:14 PM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 750 ms
Jul 14, 2010 6:41:14 PM org.apache.catalina.core.StandardService start

INFO: Starting service Catalina
Jul 14, 2010 6:41:14 PM org.apache.catalina.core.StandardEngine start
INFO: Starting Servlet Engine: Apache Tomcat/6.0.18
Jul 14, 2010 6:41:14 PM org.apache.catalina.startup.HostConfig deployWAR

INFO: Deploying web application archive dhis.war
Jul 14, 2010 6:41:16 PM org.apache.catalina.core.ApplicationContext log
INFO: Initializing Spring root WebApplicationContext

  • INFO 18:41:28,013 System property dhis2.home not set (DefaultLocationManager.java [main])

  • INFO 18:41:28,026 Environment variable DHIS2_HOME points to /home/tomcatjason (DefaultLocationManager.java [main])

orderByFragment(order-by)
sortSpecification({sort specification})
sortKeySpecification(sort key)

> sortKey($PlaceHolder$.stageinprogram)
< sortKey(null)

< sortKeySpecification(null)
< sortSpecification(null)
< orderByFragment(null)

orderByFragment(order-by)
sortSpecification({sort specification})

sortKeySpecification(sort key)
sortKey($PlaceHolder$.stageinprogram)
< sortKey(null)
< sortKeySpecification(null)
< sortSpecification(null)
< orderByFragment(null)

orderByFragment(order-by)
sortSpecification({sort specification})
sortKeySpecification(sort key)
sortKey($PlaceHolder$.sortorder)
< sortKey(null)
< sortKeySpecification(null)

< sortSpecification(null)
< orderByFragment(null)

  • INFO 18:41:59,621 Executing startup routine [1 of 11, runlevel 1]: TableAlteror (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:00,555 Tables updated (TableAlteror.java [main])

  • INFO 18:42:00,580 Executing startup routine [2 of 11, runlevel 2]: PeriodTypePopulator (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:00,651 Executing startup routine [3 of 11, runlevel 3]: TableCreator (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:00,718 Table aggregateddatavalue exists (TableCreator.java [main])

  • INFO 18:42:00,721 Table aggregatedindicatorvalue exists (TableCreator.java [main])

  • INFO 18:42:00,724 Index crosstab exists on table datavalue (TableCreator.java [main])

  • INFO 18:42:00,727 Table aggregateddatasetcompleteness exists (TableCreator.java [main])

  • INFO 18:42:00,730 Table datavaluearchive exists (TableCreator.java [main])

  • INFO 18:42:00,730 Executing startup routine [4 of 11, runlevel 3]: DataElementDefaultDimensionPopulator (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:03,355 Executing startup routine [5 of 11, runlevel 3]: MultiDimensionExpressionUpgrader (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:03,392 Executing startup routine [6 of 11, runlevel 5]: DataSetShortNamePopulator (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:03,428 Populated DataSet short name and code (DataSetShortNamePopulator.java [main])

  • INFO 18:42:03,429 Executing startup routine [7 of 11, runlevel 5]: UuidPopulator (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:03,434 Checked CategoryOption uuids (UuidPopulator.java [main])

  • INFO 18:42:03,439 Checked Category uuids (UuidPopulator.java [main])

  • INFO 18:42:04,887 Checked DataElement uuids (UuidPopulator.java [main])

  • INFO 18:42:04,888 Executing startup routine [8 of 11, runlevel 5]: OrganisationUnitGroupSetPopulator (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:04,898 Executing startup routine [9 of 11, runlevel 5]: DataEntryFormPopulator (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:04,899 Removing datasetid column from dataentryform table (DataEntryFormPopulator.java [main])

  • INFO 18:42:04,903 Executing startup routine [10 of 11, runlevel 6]: OpenHealthDataSourceWriter (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:04,904 Environment variable OPENHEALTH_HOME not set (OpenHealthDataSourceWriter.java [main])

  • INFO 18:42:04,904 Executing startup routine [11 of 11, runlevel 8]: I18nUpgrader (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:05,365 All startup routines done (DefaultStartupRoutineExecutor.java [main])
    Jul 14, 2010 6:42:13 PM org.apache.catalina.core.ApplicationContext log
    INFO: ContextListener: contextInitialized()
    Jul 14, 2010 6:42:13 PM org.apache.catalina.core.ApplicationContext log

INFO: SessionListener: contextInitialized()
Jul 14, 2010 6:42:13 PM org.apache.coyote.http11.Http11Protocol start
INFO: Starting Coyote HTTP/1.1 on http-6080
Jul 14, 2010 6:42:13 PM org.apache.jk.common.ChannelSocket init

INFO: Port busy 7009 java.net.BindException: Address already in use
Jul 14, 2010 6:42:13 PM org.apache.jk.common.ChannelSocket init
INFO: JK: ajp13 listening on /0.0.0.0:7010

Jul 14, 2010 6:42:13 PM org.apache.jk.server.JkMain start

INFO: Jk running ID=1 time=0/37 config=null
Jul 14, 2010 6:42:13 PM org.apache.catalina.startup.Catalina start
INFO: Server startup in 58789 ms

  • INFO 18:43:45,108 User login success: ‘admin’ (DefaultUserAuditService.java [http-6080-2])

  • INFO 18:56:39,905 User login success: ‘admin’ (DefaultUserAuditService.java [http-6080-3])
    ========================================================================

See guys I am starting to feel bad. I am noticing problems that none is reporting neither confirm. So I admit there is something wrong with me… should I search a medical help? :expressionless:

Caveman

On Wed, Jul 14, 2010 at 7:06 PM, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Caveman,
Not doubting you but I have not seen DHIS start without some more information.

You see nothing similar to this when DHIS starts up?

  • INFO 11:15:58,359 Executing startup routine [1 of 13, runlevel 0]: DataSetSho

rtNamePopulator (DefaultStartupRoutineExecutor.java [main])

  • INFO 11:15:58,529 Populated DataSet short name and code (DataSetShortNamePopu
    lator.java [main])
  • INFO 11:15:58,529 Executing startup routine [2 of 13, runlevel 0]: UuidPopula

tor (DefaultStartupRoutineExecutor.java [main])

  • INFO 11:15:58,536 Checked CategoryOption uuids (UuidPopulator.java [main])

  • INFO 11:15:58,543 Checked Category uuids (UuidPopulator.java [main])

  • INFO 11:15:58,561 Checked DataElement uuids (UuidPopulator.java [main])

  • INFO 11:15:58,561 Executing startup routine [3 of 13, runlevel 0]: Organisati
    onUnitHierarchyVerifier (DefaultStartupRoutineExecutor.java [main])

When you replaced the DHIS war file, did you also remove the dhis directory (rm -rf dhis) ?

Regards,
Jason

On Wed, Jul 14, 2010 at 6:24 PM, Orvalho Augusto orvaquim@gmail.com wrote:

I am saying the truth.

This is what I have on my catalina.out:

  • INFO 16:31:50,284 User login success: ‘admin’ (DefaultUserAuditService.java [http-8081-3])
    Jul 14, 2010 4:31:50 PM org.apache.velocity.runtime.log.JdkLogChute log

SEVERE: #parse() null argument

Caveman

On Wed, Jul 14, 2010 at 5:35 PM, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Caveman,
There must be something being logged in apache-tomcat-X.X.XX/logs directory. Open up a terminal, and then do tail -f catalina.out and in a separate terminal, start Tomcat. Post the result and we may be able to help.

Let me know if something is not clear.

Regards,
Jason

On Wed, Jul 14, 2010 at 4:40 PM, Orvalho Augusto orvaquim@gmail.com wrote:

There is nothing being logged.

There are no views.

Caveman

On Wed, Jul 14, 2010 at 9:03 AM, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Caveman,

Can you please post a stack trace of your application (use

pastebin.com) from startup until you start the datamart/report table

process?

Do you have any external views that are linked to the report table you

are attempting to aggregate?

Regards,

Jason

On Wed, Jul 14, 2010 at 8:57 AM, Orvalho Augusto orvaquim@gmail.com wrote:

And even do those resource tables on Postgres it does not aggregate.

Caveman

On Wed, Jul 14, 2010 at 6:35 AM, Orvalho Augusto orvaquim@gmail.com wrote:

I have changed the database (from MySQL to Postgres)… and used DHIS

tools to export and import the data.

Well, those resource tables are being built in seconds on Postgres now. So

I think there is a bug with MySQL. And I want to help to correct it.

One verry bad consequence of not importing the dataelementids is: The

entering forms does not work no more. We have to redesign the forms after an

import.

Caveman

On Tue, Jul 13, 2010 at 10:28 PM, Orvalho Augusto orvaquim@gmail.com > > > > > > > > > > > > > > > > >> wrote:

Ok.

On Tue, Jul 13, 2010 at 11:55 AM, Bob Jolliffe bobjolliffe@gmail.com > > > > > > > > > > > > > > > > >>> wrote:

On 13 July 2010 07:17, Orvalho Augusto orvaquim@gmail.com wrote:

I recompiled DHIS2.05 yesterday with help of a magician Jason

yesterday. And

the first thing I tried was this aggregation thing.

So I started redoing the orgunitstructure. It gave the same error.

While it was running I used the

show full processlist

to find out that it is a verry long insert to be run at once. I have

1664

orgunits. So it sounds like indexing trouble.

As there is a faith and “antimysql fight” I am moving my data to

postgres. I

have to do it manually.

There is no mysql fight. Just more people using postgres I guess.

I have to do it manually because DHIS does not keep de ids fixed

(organisationunid, dataelementid etc). That affects our mapping data

for

data importation from other systems. So if someone could suggest a

good

strategy is welcomed.

How are you doing mapping data from other systems? If you tell me a

bit about these systems and the data they have maybe we can make a

plan :slight_smile:

Basically these systems are:

The dataset (the DHIS concept) is represented by a table (in EpiInfo

6.06, DBF or MSAccess) where each record should have a month (or week or

quarter), year, province, district and health center columns - sometimes

they place the name of the association or a special sector of the health

center - then the other columns define a dataelement for DHIS.

So what we were doing is:

  1. Migrate the data from anywhere (EpiInfo, MSAccess and DBF) to MySQL.
  1. On MySQL we have tables where the columns names of those tables

mapping to the dataelementid (which Lars recommend not be like this)

  1. There is a script on Python made to run routinely to process this data

and insert the new one on DHIS 2 tables.

When I say routinely, I mean each month or whenever I want it.

The objective were to use solely DHIS for report and one day for data

entry.

Caveman

Cheers

Bob

Thanks for your patience guys

Caveman

2010/7/12 Jason Pickering jason.p.pickering@gmail.com

Hi Lars,

Well, yes and no. If the resource tables (namely orgunitstructure) is

not generated, it would not be possible to assign any orgunits to a

given data mart/report table? Likewise, if the orgunit levels have

not

been defined, it is also not possible to assign any orgunits to a

given report table/datamart, or perhaps I am just hallucinating ? I

have run into this problem several times actually when creating new

databases. Ideally, the orgunitstructure table should be regenerated

when there have been changes to the orgunit hierarchy and users

should

be warned that it is not possible to create a report table/datamart

without first defining orgunit level names.

Regards,

Jason

2010/7/12 Lars Helge Øverland larshelge@gmail.com:

Generating any of the resource tables don’t affect the aggregation

/

report tables…

On Sun, Jul 11, 2010 at 5:36 PM, Orvalho Augusto > > > > > > > > > > > > > > > > >>>> >> > orvaquim@gmail.com > > > > > > > > > > > > > > > > >>>> >> > wrote:

Thanks!!!

Caveman

On Sun, Jul 11, 2010 at 5:03 PM, Jason Pickering > > > > > > > > > > > > > > > > >>>> >> >> jason.p.pickering@gmail.com wrote:

A very common mistake is not having generated the resource

tables,

namely “orgunitstructure”. Consult the documentation for more

details

on how to generate these tables.

Regards,

Jason

On Sun, Jul 11, 2010 at 4:27 PM, Orvalho Augusto > > > > > > > > > > > > > > > > >>>> >> >>> orvaquim@gmail.com > > > > > > > > > > > > > > > > >>>> >> >>> wrote:

I have a dataelement collected at a Health Centre level (eg:

ats_posfem)

in

one province and only in some particular districts. In

unpredictible

future

at National level we will be collecting on every health centre.

Now, my ignorance: How can I obtain the provincial total on

Report

Table? I

have been battled with DHIS so far and nothing.

Thanks guys

Caveman


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190


Jason P. Pickering
email: jason.p.pickering@gmail.com

tel:+260968395190


Jason P. Pickering
email: jason.p.pickering@gmail.com

tel:+260968395190


Jason P. Pickering
email: jason.p.pickering@gmail.com

tel:+260968395190

Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+260968395190

I have exactly the same 1769 sourceids as the 1769 organisationunitids.

Just to resume the facts:

  1. My problem is, I can have the indicators/datalements summed if on the report I choose an parent orgnaisatiounitid.

Eg. For Niassa province I want to have all the districs summed at a certain month and get it on the table. Next time I come and choose the National level (and DHIS should sum everything down up to National). This is not happening.

  1. I was told that to have this working I should generate the resources tables.

  2. When I generated the resources tables under MySQL it never finishes and when I check with “show full processlist” I see a big INSERT running… last time I had patience it did not end even 12 hours later;

Under postgres the resources tables are generated but no objective 1.

  1. Jason asked for logs; I went to my Tomcat and there is nothing special being logged.

  2. Lars suggested to move to DHIS2.05… I went

So I am waiting for a magician. I am praying. Please God help us. :slight_smile:

Caveman

···

On Wed, Jul 14, 2010 at 9:36 PM, Jason Pickering jason.p.pickering@gmail.com wrote:

Without the logs when aggregation takes place, it will be difficult to determine what is happening.

Be sure that there are no sourceids in the “source” table without corresponding organisationanunitids in the organisationunit table.

This has caused problems for yours truly in the past. Trawl the mailing list for more information on this.

Regards,
JPP

On Wed, Jul 14, 2010 at 8:43 PM, Orvalho Augusto orvaquim@gmail.com wrote:

Is all I have on the logs.

The thing it never finishs to run the creation of the organisation structure. And when I check MySQL is occupided running a very long INSERT query. Last time I left it from 12 hours and nothing happened.

Thanks

Caveman

On Wed, Jul 14, 2010 at 8:03 PM, Jason Pickering jason.p.pickering@gmail.com wrote:

No need for medical help. I have had many of these sorts of problems as well, so you are not alone.

Can you post the log when you begin the aggregation process?

Regards,
Jason

On Wed, Jul 14, 2010 at 7:40 PM, Orvalho Augusto orvaquim@gmail.com wrote:

I understood that you want since I click on the create resource tables.

This is what since tomcat starts begin I have:

Jul 14, 2010 6:41:14 PM org.apache.catalina.core.AprLifecycleListener init

INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /home/jfclere/jakarta-tomcat-co
nnectors/jni/native/.libs
Jul 14, 2010 6:41:14 PM org.apache.coyote.http11.Http11Protocol init

INFO: Initializing Coyote HTTP/1.1 on http-6080
Jul 14, 2010 6:41:14 PM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 750 ms
Jul 14, 2010 6:41:14 PM org.apache.catalina.core.StandardService start

INFO: Starting service Catalina
Jul 14, 2010 6:41:14 PM org.apache.catalina.core.StandardEngine start
INFO: Starting Servlet Engine: Apache Tomcat/6.0.18
Jul 14, 2010 6:41:14 PM org.apache.catalina.startup.HostConfig deployWAR

INFO: Deploying web application archive dhis.war
Jul 14, 2010 6:41:16 PM org.apache.catalina.core.ApplicationContext log
INFO: Initializing Spring root WebApplicationContext

  • INFO 18:41:28,013 System property dhis2.home not set (DefaultLocationManager.java [main])

  • INFO 18:41:28,026 Environment variable DHIS2_HOME points to /home/tomcatjason (DefaultLocationManager.java [main])

orderByFragment(order-by)
sortSpecification({sort specification})
sortKeySpecification(sort key)

> sortKey($PlaceHolder$.stageinprogram)
< sortKey(null)

< sortKeySpecification(null)
< sortSpecification(null)
< orderByFragment(null)

orderByFragment(order-by)
sortSpecification({sort specification})

sortKeySpecification(sort key)
sortKey($PlaceHolder$.stageinprogram)
< sortKey(null)
< sortKeySpecification(null)
< sortSpecification(null)
< orderByFragment(null)

orderByFragment(order-by)
sortSpecification({sort specification})
sortKeySpecification(sort key)
sortKey($PlaceHolder$.sortorder)
< sortKey(null)
< sortKeySpecification(null)

< sortSpecification(null)
< orderByFragment(null)

  • INFO 18:41:59,621 Executing startup routine [1 of 11, runlevel 1]: TableAlteror (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:00,555 Tables updated (TableAlteror.java [main])

  • INFO 18:42:00,580 Executing startup routine [2 of 11, runlevel 2]: PeriodTypePopulator (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:00,651 Executing startup routine [3 of 11, runlevel 3]: TableCreator (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:00,718 Table aggregateddatavalue exists (TableCreator.java [main])

  • INFO 18:42:00,721 Table aggregatedindicatorvalue exists (TableCreator.java [main])

  • INFO 18:42:00,724 Index crosstab exists on table datavalue (TableCreator.java [main])

  • INFO 18:42:00,727 Table aggregateddatasetcompleteness exists (TableCreator.java [main])

  • INFO 18:42:00,730 Table datavaluearchive exists (TableCreator.java [main])

  • INFO 18:42:00,730 Executing startup routine [4 of 11, runlevel 3]: DataElementDefaultDimensionPopulator (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:03,355 Executing startup routine [5 of 11, runlevel 3]: MultiDimensionExpressionUpgrader (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:03,392 Executing startup routine [6 of 11, runlevel 5]: DataSetShortNamePopulator (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:03,428 Populated DataSet short name and code (DataSetShortNamePopulator.java [main])

  • INFO 18:42:03,429 Executing startup routine [7 of 11, runlevel 5]: UuidPopulator (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:03,434 Checked CategoryOption uuids (UuidPopulator.java [main])

  • INFO 18:42:03,439 Checked Category uuids (UuidPopulator.java [main])

  • INFO 18:42:04,887 Checked DataElement uuids (UuidPopulator.java [main])

  • INFO 18:42:04,888 Executing startup routine [8 of 11, runlevel 5]: OrganisationUnitGroupSetPopulator (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:04,898 Executing startup routine [9 of 11, runlevel 5]: DataEntryFormPopulator (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:04,899 Removing datasetid column from dataentryform table (DataEntryFormPopulator.java [main])

  • INFO 18:42:04,903 Executing startup routine [10 of 11, runlevel 6]: OpenHealthDataSourceWriter (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:04,904 Environment variable OPENHEALTH_HOME not set (OpenHealthDataSourceWriter.java [main])

  • INFO 18:42:04,904 Executing startup routine [11 of 11, runlevel 8]: I18nUpgrader (DefaultStartupRoutineExecutor.java [main])

  • INFO 18:42:05,365 All startup routines done (DefaultStartupRoutineExecutor.java [main])
    Jul 14, 2010 6:42:13 PM org.apache.catalina.core.ApplicationContext log
    INFO: ContextListener: contextInitialized()
    Jul 14, 2010 6:42:13 PM org.apache.catalina.core.ApplicationContext log

INFO: SessionListener: contextInitialized()
Jul 14, 2010 6:42:13 PM org.apache.coyote.http11.Http11Protocol start
INFO: Starting Coyote HTTP/1.1 on http-6080
Jul 14, 2010 6:42:13 PM org.apache.jk.common.ChannelSocket init

INFO: Port busy 7009 java.net.BindException: Address already in use
Jul 14, 2010 6:42:13 PM org.apache.jk.common.ChannelSocket init
INFO: JK: ajp13 listening on /0.0.0.0:7010

Jul 14, 2010 6:42:13 PM org.apache.jk.server.JkMain start

INFO: Jk running ID=1 time=0/37 config=null
Jul 14, 2010 6:42:13 PM org.apache.catalina.startup.Catalina start
INFO: Server startup in 58789 ms

  • INFO 18:43:45,108 User login success: ‘admin’ (DefaultUserAuditService.java [http-6080-2])

  • INFO 18:56:39,905 User login success: ‘admin’ (DefaultUserAuditService.java [http-6080-3])
    ========================================================================

See guys I am starting to feel bad. I am noticing problems that none is reporting neither confirm. So I admit there is something wrong with me… should I search a medical help? :expressionless:

Caveman

On Wed, Jul 14, 2010 at 7:06 PM, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Caveman,
Not doubting you but I have not seen DHIS start without some more information.

You see nothing similar to this when DHIS starts up?

  • INFO 11:15:58,359 Executing startup routine [1 of 13, runlevel 0]: DataSetSho

rtNamePopulator (DefaultStartupRoutineExecutor.java [main])

  • INFO 11:15:58,529 Populated DataSet short name and code (DataSetShortNamePopu
    lator.java [main])
  • INFO 11:15:58,529 Executing startup routine [2 of 13, runlevel 0]: UuidPopula

tor (DefaultStartupRoutineExecutor.java [main])

  • INFO 11:15:58,536 Checked CategoryOption uuids (UuidPopulator.java [main])

  • INFO 11:15:58,543 Checked Category uuids (UuidPopulator.java [main])

  • INFO 11:15:58,561 Checked DataElement uuids (UuidPopulator.java [main])

  • INFO 11:15:58,561 Executing startup routine [3 of 13, runlevel 0]: Organisati
    onUnitHierarchyVerifier (DefaultStartupRoutineExecutor.java [main])

When you replaced the DHIS war file, did you also remove the dhis directory (rm -rf dhis) ?

Regards,
Jason

On Wed, Jul 14, 2010 at 6:24 PM, Orvalho Augusto orvaquim@gmail.com wrote:

I am saying the truth.

This is what I have on my catalina.out:

  • INFO 16:31:50,284 User login success: ‘admin’ (DefaultUserAuditService.java [http-8081-3])
    Jul 14, 2010 4:31:50 PM org.apache.velocity.runtime.log.JdkLogChute log

SEVERE: #parse() null argument

Caveman

On Wed, Jul 14, 2010 at 5:35 PM, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Caveman,
There must be something being logged in apache-tomcat-X.X.XX/logs directory. Open up a terminal, and then do tail -f catalina.out and in a separate terminal, start Tomcat. Post the result and we may be able to help.

Let me know if something is not clear.

Regards,
Jason

On Wed, Jul 14, 2010 at 4:40 PM, Orvalho Augusto orvaquim@gmail.com wrote:

There is nothing being logged.

There are no views.

Caveman

On Wed, Jul 14, 2010 at 9:03 AM, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Caveman,

Can you please post a stack trace of your application (use

pastebin.com) from startup until you start the datamart/report table

process?

Do you have any external views that are linked to the report table you

are attempting to aggregate?

Regards,

Jason

On Wed, Jul 14, 2010 at 8:57 AM, Orvalho Augusto orvaquim@gmail.com wrote:

And even do those resource tables on Postgres it does not aggregate.

Caveman

On Wed, Jul 14, 2010 at 6:35 AM, Orvalho Augusto orvaquim@gmail.com wrote:

I have changed the database (from MySQL to Postgres)… and used DHIS

tools to export and import the data.

Well, those resource tables are being built in seconds on Postgres now. So

I think there is a bug with MySQL. And I want to help to correct it.

One verry bad consequence of not importing the dataelementids is: The

entering forms does not work no more. We have to redesign the forms after an

import.

Caveman

On Tue, Jul 13, 2010 at 10:28 PM, Orvalho Augusto orvaquim@gmail.com > > > > > > > > > > > > > > > > > > >> wrote:

Ok.

On Tue, Jul 13, 2010 at 11:55 AM, Bob Jolliffe bobjolliffe@gmail.com > > > > > > > > > > > > > > > > > > >>> wrote:

On 13 July 2010 07:17, Orvalho Augusto orvaquim@gmail.com wrote:

I recompiled DHIS2.05 yesterday with help of a magician Jason

yesterday. And

the first thing I tried was this aggregation thing.

So I started redoing the orgunitstructure. It gave the same error.

While it was running I used the

show full processlist

to find out that it is a verry long insert to be run at once. I have

1664

orgunits. So it sounds like indexing trouble.

As there is a faith and “antimysql fight” I am moving my data to

postgres. I

have to do it manually.

There is no mysql fight. Just more people using postgres I guess.

I have to do it manually because DHIS does not keep de ids fixed

(organisationunid, dataelementid etc). That affects our mapping data

for

data importation from other systems. So if someone could suggest a

good

strategy is welcomed.

How are you doing mapping data from other systems? If you tell me a

bit about these systems and the data they have maybe we can make a

plan :slight_smile:

Basically these systems are:

The dataset (the DHIS concept) is represented by a table (in EpiInfo

6.06, DBF or MSAccess) where each record should have a month (or week or

quarter), year, province, district and health center columns - sometimes

they place the name of the association or a special sector of the health

center - then the other columns define a dataelement for DHIS.

So what we were doing is:

  1. Migrate the data from anywhere (EpiInfo, MSAccess and DBF) to MySQL.
  1. On MySQL we have tables where the columns names of those tables

mapping to the dataelementid (which Lars recommend not be like this)

  1. There is a script on Python made to run routinely to process this data

and insert the new one on DHIS 2 tables.

When I say routinely, I mean each month or whenever I want it.

The objective were to use solely DHIS for report and one day for data

entry.

Caveman

Cheers

Bob

Thanks for your patience guys

Caveman

2010/7/12 Jason Pickering jason.p.pickering@gmail.com

Hi Lars,

Well, yes and no. If the resource tables (namely orgunitstructure) is

not generated, it would not be possible to assign any orgunits to a

given data mart/report table? Likewise, if the orgunit levels have

not

been defined, it is also not possible to assign any orgunits to a

given report table/datamart, or perhaps I am just hallucinating ? I

have run into this problem several times actually when creating new

databases. Ideally, the orgunitstructure table should be regenerated

when there have been changes to the orgunit hierarchy and users

should

be warned that it is not possible to create a report table/datamart

without first defining orgunit level names.

Regards,

Jason

2010/7/12 Lars Helge Øverland larshelge@gmail.com:

Generating any of the resource tables don’t affect the aggregation

/

report tables…

On Sun, Jul 11, 2010 at 5:36 PM, Orvalho Augusto > > > > > > > > > > > > > > > > > > >>>> >> > orvaquim@gmail.com > > > > > > > > > > > > > > > > > > >>>> >> > wrote:

Thanks!!!

Caveman

On Sun, Jul 11, 2010 at 5:03 PM, Jason Pickering > > > > > > > > > > > > > > > > > > >>>> >> >> jason.p.pickering@gmail.com wrote:

A very common mistake is not having generated the resource

tables,

namely “orgunitstructure”. Consult the documentation for more

details

on how to generate these tables.

Regards,

Jason

On Sun, Jul 11, 2010 at 4:27 PM, Orvalho Augusto > > > > > > > > > > > > > > > > > > >>>> >> >>> orvaquim@gmail.com > > > > > > > > > > > > > > > > > > >>>> >> >>> wrote:

I have a dataelement collected at a Health Centre level (eg:

ats_posfem)

in

one province and only in some particular districts. In

unpredictible

future

at National level we will be collecting on every health centre.

Now, my ignorance: How can I obtain the provincial total on

Report

Table? I

have been battled with DHIS so far and nothing.

Thanks guys

Caveman


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190


Jason P. Pickering
email: jason.p.pickering@gmail.com

tel:+260968395190


Jason P. Pickering
email: jason.p.pickering@gmail.com

tel:+260968395190


Jason P. Pickering
email: jason.p.pickering@gmail.com

tel:+260968395190


Jason P. Pickering
email: jason.p.pickering@gmail.com

tel:+260968395190