Aggregating values

Support - Assistance technique
1

I have a dataelement collected at a Health Centre level (eg: ats_posfem) in one province and only in some particular districts. In unpredictible future at National level we will be collecting on every health centre.

Now, my ignorance: How can I obtain the provincial total on Report Table? I have been battled with DHIS so far and nothing.

Thanks guys
Caveman

2

A very common mistake is not having generated the resource tables,
namely "orgunitstructure". Consult the documentation for more details
on how to generate these tables.

Regards,
Jason

···

On Sun, Jul 11, 2010 at 4:27 PM, Orvalho Augusto <orvaquim@gmail.com> wrote:

I have a dataelement collected at a Health Centre level (eg: ats_posfem) in
one province and only in some particular districts. In unpredictible future
at National level we will be collecting on every health centre.

Now, my ignorance: How can I obtain the provincial total on Report Table? I
have been battled with DHIS so far and nothing.

Thanks guys
Caveman

_______________________________________________
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help : https://help.launchpad.net/ListHelp

--
--
Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+260968395190

3

Thanks!!!

Caveman

···

On Sun, Jul 11, 2010 at 5:03 PM, Jason Pickering jason.p.pickering@gmail.com wrote:

A very common mistake is not having generated the resource tables,

namely “orgunitstructure”. Consult the documentation for more details

on how to generate these tables.

Regards,

Jason

On Sun, Jul 11, 2010 at 4:27 PM, Orvalho Augusto orvaquim@gmail.com wrote:

I have a dataelement collected at a Health Centre level (eg: ats_posfem) in

one province and only in some particular districts. In unpredictible future

at National level we will be collecting on every health centre.

Now, my ignorance: How can I obtain the provincial total on Report Table? I

have been battled with DHIS so far and nothing.

Thanks guys

Caveman

Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

–

–

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190

4

I did not have success. My logs are:

···

===========================================================

  • ERROR 21:24:31,306 Error while executing action (ExceptionInterceptor.java [http-8081-6])
    java.lang.RuntimeException: Failed to flush BatchHandler

    at org.amplecode.quick.batchhandler.AbstractBatchHandler.flush(AbstractBatchHandler.java:290)
    at org.hisp.dhis.resourcetable.DefaultResourceTableService.generateOrganisationUnitStructures(DefaultResourceTableService.java:196)

    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

    at java.lang.reflect.Method.invoke(Method.java:616)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)

    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:106)

    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
    at $Proxy21.generateOrganisationUnitStructures(Unknown Source)

    at org.hisp.dhis.dataadmin.action.resourcetable.GenerateResourceTableAction.execute(GenerateResourceTableAction.java:114)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:616)
    at com.opensymphony.xwork2.DefaultActionInvocation.invokeAction(DefaultActionInvocation.java:440)

    at com.opensymphony.xwork2.DefaultActionInvocation.invokeActionOnly(DefaultActionInvocation.java:279)
    at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:242)
    at org.hisp.dhis.webportal.interceptor.XWorkPortalUserInterceptor.intercept(XWorkPortalUserInterceptor.java:82)

    at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
    at org.hisp.dhis.webportal.interceptor.XWorkPortalModuleInterceptor.intercept(XWorkPortalModuleInterceptor.java:85)

    at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
    at org.hisp.dhis.webportal.interceptor.XWorkPortalMenuInterceptor.intercept(XWorkPortalMenuInterceptor.java:91)
    at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)

    at org.hisp.dhis.webportal.interceptor.XWorkPortalParamsInterceptor.intercept(XWorkPortalParamsInterceptor.java:117)
    at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)

    at org.hisp.dhis.interceptor.StyleInterceptor.intercept(StyleInterceptor.java:89)
    at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
    at org.hisp.dhis.interceptor.SystemSettingInterceptor.intercept(SystemSettingInterceptor.java:94)

    at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
    at org.hisp.dhis.interceptor.DataDictionaryModeInterceptor.intercept(DataDictionaryModeInterceptor.java:109)
    at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)

    at org.hisp.dhis.interceptor.DisplayPropertyInterceptor.intercept(DisplayPropertyInterceptor.java:90)
    at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
    at org.hisp.dhis.interceptor.SortOrderInterceptor.intercept(SortOrderInterceptor.java:124)

    at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
    at org.hisp.dhis.security.intercept.XWorkSecurityInterceptor.intercept(XWorkSecurityInterceptor.java:98)
    at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)

    at org.hisp.dhis.interceptor.I18nInterceptor.intercept(I18nInterceptor.java:141)
    at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
    at com.opensymphony.xwork2.interceptor.ChainingInterceptor.intercept(ChainingInterceptor.java:126)

    at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
    at com.opensymphony.xwork2.interceptor.ParametersInterceptor.doIntercept(ParametersInterceptor.java:195)
    at com.opensymphony.xwork2.interceptor.MethodFilterInterceptor.intercept(MethodFilterInterceptor.java:87)

    at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
    at com.opensymphony.xwork2.interceptor.StaticParametersInterceptor.intercept(StaticParametersInterceptor.java:148)

    at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
    at org.hisp.dhis.interceptor.ExceptionInterceptor.intercept(ExceptionInterceptor.java:89)
    at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)

    at org.apache.struts2.impl.StrutsActionProxy.execute(StrutsActionProxy.java:52)
    at org.apache.struts2.dispatcher.Dispatcher.serviceAction(Dispatcher.java:468)
    at org.apache.struts2.dispatcher.FilterDispatcher.doFilter(FilterDispatcher.java:395)

    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:378)

    at org.hisp.dhis.security.filter.RequiredLoginFilter.doFilter(RequiredLoginFilter.java:129)
    at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:390)
    at org.hisp.dhis.security.filter.AutomaticAccessFilter.doFilter(AutomaticAccessFilter.java:112)

    at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:390)
    at org.springframework.security.ui.logout.LogoutFilter.doFilterHttp(LogoutFilter.java:89)
    at org.springframework.security.ui.SpringSecurityFilter.doFilter(SpringSecurityFilter.java:53)

    at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:390)
    at org.springframework.security.ui.AbstractProcessingFilter.doFilterHttp(AbstractProcessingFilter.java:278)

    at org.springframework.security.ui.SpringSecurityFilter.doFilter(SpringSecurityFilter.java:53)
    at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:390)
    at org.springframework.security.context.HttpSessionContextIntegrationFilter.doFilterHttp(HttpSessionContextIntegrationFilter.java:235)

    at org.springframework.security.ui.SpringSecurityFilter.doFilter(SpringSecurityFilter.java:53)
    at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:390)
    at org.springframework.security.util.FilterChainProxy.doFilter(FilterChainProxy.java:175)

    at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:236)
    at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)

    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.springframework.orm.hibernate3.support.OpenSessionInViewFilter.doFilterInternal(OpenSessionInViewFilter.java:198)

    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)

    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)

    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)

    at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:849)
    at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
    at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:454)

    at java.lang.Thread.run(Thread.java:636)
    Caused by: java.sql.SQLException: Lock wait timeout exceeded; try restarting transaction
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1055)
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:956)

    at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3491)
    at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3423)
    at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1936)
    at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2060)

    at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2536)
    at com.mysql.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1564)
    at com.mysql.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1485)

    at org.amplecode.quick.batchhandler.AbstractBatchHandler.flush(AbstractBatchHandler.java:274)
    … 89 more

==================================================================

This seems to come again from database thing.

Caveman

On Sun, Jul 11, 2010 at 5:03 PM, Jason Pickering jason.p.pickering@gmail.com wrote:

A very common mistake is not having generated the resource tables,

namely “orgunitstructure”. Consult the documentation for more details

on how to generate these tables.

Regards,

Jason

On Sun, Jul 11, 2010 at 4:27 PM, Orvalho Augusto orvaquim@gmail.com wrote:

I have a dataelement collected at a Health Centre level (eg: ats_posfem) in

one province and only in some particular districts. In unpredictible future

at National level we will be collecting on every health centre.

Now, my ignorance: How can I obtain the provincial total on Report Table? I

have been battled with DHIS so far and nothing.

Thanks guys

Caveman

Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

–

–

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190

5

Generating any of the resource tables don't affect the aggregation /
report tables...

···

On Sun, Jul 11, 2010 at 5:36 PM, Orvalho Augusto <orvaquim@gmail.com> wrote:

Thanks!!!

Caveman

On Sun, Jul 11, 2010 at 5:03 PM, Jason Pickering > <jason.p.pickering@gmail.com> wrote:

A very common mistake is not having generated the resource tables,
namely "orgunitstructure". Consult the documentation for more details
on how to generate these tables.

Regards,
Jason

On Sun, Jul 11, 2010 at 4:27 PM, Orvalho Augusto <orvaquim@gmail.com> >> wrote:
> I have a dataelement collected at a Health Centre level (eg: ats_posfem)
> in
> one province and only in some particular districts. In unpredictible
> future
> at National level we will be collecting on every health centre.
>
> Now, my ignorance: How can I obtain the provincial total on Report
> Table? I
> have been battled with DHIS so far and nothing.
>
> Thanks guys
> Caveman
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~dhis2-devs
> Post to : dhis2-devs@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~dhis2-devs
> More help : https://help.launchpad.net/ListHelp
>
>

--
--
Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+260968395190

_______________________________________________
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help : https://help.launchpad.net/ListHelp

6

Guys I found why.

It is because generating organitstructure takes a long time. (Mytop monitored this query: INSERT INTO orgunitstructure …)

The default transaction waiting time on MySQL is 50seconds.
I increased this innodb_lock_wait_timeout to 600seconds and it runs.

Please someone can help me. I am get a sensation that the DHIS community is abandoning MySQL support. Is it that true?

Caveman

···

On Sun, Jul 11, 2010 at 9:28 PM, Orvalho Augusto orvaquim@gmail.com wrote:

I did not have success. My logs are:

===========================================================

  • ERROR 21:24:31,306 Error while executing action (ExceptionInterceptor.java [http-8081-6])
    java.lang.RuntimeException: Failed to flush BatchHandler

    at org.amplecode.quick.batchhandler.AbstractBatchHandler.flush(AbstractBatchHandler.java:290)
    at org.hisp.dhis.resourcetable.DefaultResourceTableService.generateOrganisationUnitStructures(DefaultResourceTableService.java:196)

    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

    at java.lang.reflect.Method.invoke(Method.java:616)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)

    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:106)

    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
    at $Proxy21.generateOrganisationUnitStructures(Unknown Source)

    at org.hisp.dhis.dataadmin.action.resourcetable.GenerateResourceTableAction.execute(GenerateResourceTableAction.java:114)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:616)
    at com.opensymphony.xwork2.DefaultActionInvocation.invokeAction(DefaultActionInvocation.java:440)

    at com.opensymphony.xwork2.DefaultActionInvocation.invokeActionOnly(DefaultActionInvocation.java:279)
    at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:242)
    at org.hisp.dhis.webportal.interceptor.XWorkPortalUserInterceptor.intercept(XWorkPortalUserInterceptor.java:82)

    at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
    at org.hisp.dhis.webportal.interceptor.XWorkPortalModuleInterceptor.intercept(XWorkPortalModuleInterceptor.java:85)

    at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
    at org.hisp.dhis.webportal.interceptor.XWorkPortalMenuInterceptor.intercept(XWorkPortalMenuInterceptor.java:91)
    at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)

    at org.hisp.dhis.webportal.interceptor.XWorkPortalParamsInterceptor.intercept(XWorkPortalParamsInterceptor.java:117)
    at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)

    at org.hisp.dhis.interceptor.StyleInterceptor.intercept(StyleInterceptor.java:89)
    at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
    at org.hisp.dhis.interceptor.SystemSettingInterceptor.intercept(SystemSettingInterceptor.java:94)

    at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
    at org.hisp.dhis.interceptor.DataDictionaryModeInterceptor.intercept(DataDictionaryModeInterceptor.java:109)
    at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)

    at org.hisp.dhis.interceptor.DisplayPropertyInterceptor.intercept(DisplayPropertyInterceptor.java:90)
    at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
    at org.hisp.dhis.interceptor.SortOrderInterceptor.intercept(SortOrderInterceptor.java:124)

    at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
    at org.hisp.dhis.security.intercept.XWorkSecurityInterceptor.intercept(XWorkSecurityInterceptor.java:98)
    at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)

    at org.hisp.dhis.interceptor.I18nInterceptor.intercept(I18nInterceptor.java:141)
    at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
    at com.opensymphony.xwork2.interceptor.ChainingInterceptor.intercept(ChainingInterceptor.java:126)

    at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
    at com.opensymphony.xwork2.interceptor.ParametersInterceptor.doIntercept(ParametersInterceptor.java:195)
    at com.opensymphony.xwork2.interceptor.MethodFilterInterceptor.intercept(MethodFilterInterceptor.java:87)

    at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
    at com.opensymphony.xwork2.interceptor.StaticParametersInterceptor.intercept(StaticParametersInterceptor.java:148)

    at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
    at org.hisp.dhis.interceptor.ExceptionInterceptor.intercept(ExceptionInterceptor.java:89)
    at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)

    at org.apache.struts2.impl.StrutsActionProxy.execute(StrutsActionProxy.java:52)
    at org.apache.struts2.dispatcher.Dispatcher.serviceAction(Dispatcher.java:468)
    at org.apache.struts2.dispatcher.FilterDispatcher.doFilter(FilterDispatcher.java:395)

    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:378)

    at org.hisp.dhis.security.filter.RequiredLoginFilter.doFilter(RequiredLoginFilter.java:129)
    at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:390)
    at org.hisp.dhis.security.filter.AutomaticAccessFilter.doFilter(AutomaticAccessFilter.java:112)

    at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:390)
    at org.springframework.security.ui.logout.LogoutFilter.doFilterHttp(LogoutFilter.java:89)
    at org.springframework.security.ui.SpringSecurityFilter.doFilter(SpringSecurityFilter.java:53)

    at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:390)
    at org.springframework.security.ui.AbstractProcessingFilter.doFilterHttp(AbstractProcessingFilter.java:278)

    at org.springframework.security.ui.SpringSecurityFilter.doFilter(SpringSecurityFilter.java:53)
    at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:390)
    at org.springframework.security.context.HttpSessionContextIntegrationFilter.doFilterHttp(HttpSessionContextIntegrationFilter.java:235)

    at org.springframework.security.ui.SpringSecurityFilter.doFilter(SpringSecurityFilter.java:53)
    at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:390)
    at org.springframework.security.util.FilterChainProxy.doFilter(FilterChainProxy.java:175)

    at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:236)
    at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)

    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.springframework.orm.hibernate3.support.OpenSessionInViewFilter.doFilterInternal(OpenSessionInViewFilter.java:198)

    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)

    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)

    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)

    at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:849)
    at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
    at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:454)

    at java.lang.Thread.run(Thread.java:636)
    Caused by: java.sql.SQLException: Lock wait timeout exceeded; try restarting transaction
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1055)
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:956)

    at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3491)
    at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3423)
    at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1936)
    at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2060)

    at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2536)
    at com.mysql.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1564)
    at com.mysql.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1485)

    at org.amplecode.quick.batchhandler.AbstractBatchHandler.flush(AbstractBatchHandler.java:274)
    … 89 more

==================================================================

This seems to come again from database thing.

Caveman

On Sun, Jul 11, 2010 at 5:03 PM, Jason Pickering jason.p.pickering@gmail.com wrote:

A very common mistake is not having generated the resource tables,

namely “orgunitstructure”. Consult the documentation for more details

on how to generate these tables.

Regards,

Jason

On Sun, Jul 11, 2010 at 4:27 PM, Orvalho Augusto orvaquim@gmail.com wrote:

I have a dataelement collected at a Health Centre level (eg: ats_posfem) in

one province and only in some particular districts. In unpredictible future

at National level we will be collecting on every health centre.

Now, my ignorance: How can I obtain the provincial total on Report Table? I

have been battled with DHIS so far and nothing.

Thanks guys

Caveman

Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

–

–

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190

7

I need to have a repo/table that aggregates data as I asked before. I did not find it on docs.

Someone told me that can happens if a generate those resource tables. I try to generate those tables and I get stuck on that error I reported.

Good for me if I do not need to generate such tables and ask what I should do?

Caveman

···

2010/7/12 Lars Helge Øverland larshelge@gmail.com

Generating any of the resource tables don’t affect the aggregation /

report tables…

On Sun, Jul 11, 2010 at 5:36 PM, Orvalho Augusto orvaquim@gmail.com wrote:

Thanks!!!

Caveman

On Sun, Jul 11, 2010 at 5:03 PM, Jason Pickering > > > jason.p.pickering@gmail.com wrote:

A very common mistake is not having generated the resource tables,

namely “orgunitstructure”. Consult the documentation for more details

on how to generate these tables.

Regards,

Jason

On Sun, Jul 11, 2010 at 4:27 PM, Orvalho Augusto orvaquim@gmail.com > > >> wrote:

I have a dataelement collected at a Health Centre level (eg: ats_posfem)

in

one province and only in some particular districts. In unpredictible

future

at National level we will be collecting on every health centre.

Now, my ignorance: How can I obtain the provincial total on Report

Table? I

have been battled with DHIS so far and nothing.

Thanks guys

Caveman

Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

–

–

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190

Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

8

Hi Caveman, we are definitely not dropping mysql support, its just
that most of the devs are using postgres so a function not being
mysql-compliant might sneak in from time to time. I tested most of the
application on mysql recently without trouble.

A common problem with aggregation earlier was the way the orgunit
hierarchy was represented and stored, this is changed in 2.0.5
snapshot and is way less error-prone now. So please build yourself a
war-file from trunk and test with that.

If that doesn't work please send me a database dump and I will have a
look. (On a sidenote a main reason for choosing postgres is the way
dumps are restored - mysql uses single regular sql insert statements
which might take days for very large databases while postgres uses
more "native" copy statements which never takes more than minutes).

cheers, Lars

···

2010/7/12 Orvalho Augusto <orvaquim@gmail.com>:

I need to have a repo/table that aggregates data as I asked before. I did
not find it on docs.

Someone told me that can happens if a generate those resource tables. I try
to generate those tables and I get stuck on that error I reported.

Good for me if I do not need to generate such tables and ask what I should
do?

Caveman

2010/7/12 Lars Helge Øverland <larshelge@gmail.com>

Generating any of the resource tables don't affect the aggregation /
report tables...

On Sun, Jul 11, 2010 at 5:36 PM, Orvalho Augusto <orvaquim@gmail.com> >> wrote:
> Thanks!!!
>
> Caveman
>
>
> On Sun, Jul 11, 2010 at 5:03 PM, Jason Pickering >> > <jason.p.pickering@gmail.com> wrote:
>>
>> A very common mistake is not having generated the resource tables,
>> namely "orgunitstructure". Consult the documentation for more details
>> on how to generate these tables.
>>
>> Regards,
>> Jason
>>
>>
>> On Sun, Jul 11, 2010 at 4:27 PM, Orvalho Augusto <orvaquim@gmail.com> >> >> wrote:
>> > I have a dataelement collected at a Health Centre level (eg:
>> > ats_posfem)
>> > in
>> > one province and only in some particular districts. In unpredictible
>> > future
>> > at National level we will be collecting on every health centre.
>> >
>> > Now, my ignorance: How can I obtain the provincial total on Report
>> > Table? I
>> > have been battled with DHIS so far and nothing.
>> >
>> > Thanks guys
>> > Caveman
>> >
>> >
>> > _______________________________________________
>> > Mailing list: https://launchpad.net/~dhis2-devs
>> > Post to : dhis2-devs@lists.launchpad.net
>> > Unsubscribe : https://launchpad.net/~dhis2-devs
>> > More help : https://help.launchpad.net/ListHelp
>> >
>> >
>>
>>
>>
>> --
>> --
>> Jason P. Pickering
>> email: jason.p.pickering@gmail.com
>> tel:+260968395190
>
>
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~dhis2-devs
> Post to : dhis2-devs@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~dhis2-devs
> More help : https://help.launchpad.net/ListHelp
>
>

9

Thanks!!

That of restoring a database of huge size is not everyday you do.

I need people with databases of Postgres with sizes above 5GB. And tell us how their postgres behave, specifically tell us how they address the vaccum data thing.

Please is not a war.

Caveman

···

2010/7/12 Lars Helge Øverland larshelge@gmail.com

Hi Caveman, we are definitely not dropping mysql support, its just

that most of the devs are using postgres so a function not being

mysql-compliant might sneak in from time to time. I tested most of the

application on mysql recently without trouble.

A common problem with aggregation earlier was the way the orgunit

hierarchy was represented and stored, this is changed in 2.0.5

snapshot and is way less error-prone now. So please build yourself a

war-file from trunk and test with that.

If that doesn’t work please send me a database dump and I will have a

look. (On a sidenote a main reason for choosing postgres is the way

dumps are restored - mysql uses single regular sql insert statements

which might take days for very large databases while postgres uses

more “native” copy statements which never takes more than minutes).

cheers, Lars

2010/7/12 Orvalho Augusto orvaquim@gmail.com:

I need to have a repo/table that aggregates data as I asked before. I did

not find it on docs.

Someone told me that can happens if a generate those resource tables. I try

to generate those tables and I get stuck on that error I reported.

Good for me if I do not need to generate such tables and ask what I should

do?

Caveman

2010/7/12 Lars Helge Øverland larshelge@gmail.com

Generating any of the resource tables don’t affect the aggregation /

report tables…

On Sun, Jul 11, 2010 at 5:36 PM, Orvalho Augusto orvaquim@gmail.com > > >> wrote:

Thanks!!!

Caveman

On Sun, Jul 11, 2010 at 5:03 PM, Jason Pickering > > >> > jason.p.pickering@gmail.com wrote:

A very common mistake is not having generated the resource tables,

namely “orgunitstructure”. Consult the documentation for more details

on how to generate these tables.

Regards,

Jason

On Sun, Jul 11, 2010 at 4:27 PM, Orvalho Augusto orvaquim@gmail.com > > >> >> wrote:

I have a dataelement collected at a Health Centre level (eg:

ats_posfem)

in

one province and only in some particular districts. In unpredictible

future

at National level we will be collecting on every health centre.

Now, my ignorance: How can I obtain the provincial total on Report

Table? I

have been battled with DHIS so far and nothing.

Thanks guys

Caveman

Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

–

–

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190

Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

10

Thanks!!

That of restoring a database of huge size is not everyday you do.

Well that depends :slight_smile:

I need people with databases of Postgres with sizes above 5GB. And tell us
how their postgres behave, specifically tell us how they address the vaccum
data thing.

Vacuum and analyze jobs are run as a daemon by default as far as I know.

From the postgres docs:

Maximum Database Size Unlimited
Maximum Table Size 32 TB
Maximum Row Size 1.6 TB
Maximum Field Size 1 GB
Maximum Rows per Table Unlimited
Maximum Columns per Table 250 - 1600 depending on column types
Maximum Indexes per Table Unlimited

So 5 GB should not be a problem..

Please is not a war.

Definitely not :slight_smile:

···

2010/7/12 Orvalho Augusto <orvaquim@gmail.com>:

Caveman

2010/7/12 Lars Helge Øverland <larshelge@gmail.com>

Hi Caveman, we are definitely not dropping mysql support, its just
that most of the devs are using postgres so a function not being
mysql-compliant might sneak in from time to time. I tested most of the
application on mysql recently without trouble.

A common problem with aggregation earlier was the way the orgunit
hierarchy was represented and stored, this is changed in 2.0.5
snapshot and is way less error-prone now. So please build yourself a
war-file from trunk and test with that.

If that doesn't work please send me a database dump and I will have a
look. (On a sidenote a main reason for choosing postgres is the way
dumps are restored - mysql uses single regular sql insert statements
which might take days for very large databases while postgres uses
more "native" copy statements which never takes more than minutes).

cheers, Lars

2010/7/12 Orvalho Augusto <orvaquim@gmail.com>:
> I need to have a repo/table that aggregates data as I asked before. I
> did
> not find it on docs.
>
> Someone told me that can happens if a generate those resource tables. I
> try
> to generate those tables and I get stuck on that error I reported.
>
> Good for me if I do not need to generate such tables and ask what I
> should
> do?
>
> Caveman
>
>
> 2010/7/12 Lars Helge Øverland <larshelge@gmail.com>
>>
>> Generating any of the resource tables don't affect the aggregation /
>> report tables...
>>
>>
>>
>> On Sun, Jul 11, 2010 at 5:36 PM, Orvalho Augusto <orvaquim@gmail.com> >> >> wrote:
>> > Thanks!!!
>> >
>> > Caveman
>> >
>> >
>> > On Sun, Jul 11, 2010 at 5:03 PM, Jason Pickering >> >> > <jason.p.pickering@gmail.com> wrote:
>> >>
>> >> A very common mistake is not having generated the resource tables,
>> >> namely "orgunitstructure". Consult the documentation for more
>> >> details
>> >> on how to generate these tables.
>> >>
>> >> Regards,
>> >> Jason
>> >>
>> >>
>> >> On Sun, Jul 11, 2010 at 4:27 PM, Orvalho Augusto >> >> >> <orvaquim@gmail.com> >> >> >> wrote:
>> >> > I have a dataelement collected at a Health Centre level (eg:
>> >> > ats_posfem)
>> >> > in
>> >> > one province and only in some particular districts. In
>> >> > unpredictible
>> >> > future
>> >> > at National level we will be collecting on every health centre.
>> >> >
>> >> > Now, my ignorance: How can I obtain the provincial total on Report
>> >> > Table? I
>> >> > have been battled with DHIS so far and nothing.
>> >> >
>> >> > Thanks guys
>> >> > Caveman
>> >> >
>> >> >
>> >> > _______________________________________________
>> >> > Mailing list: https://launchpad.net/~dhis2-devs
>> >> > Post to : dhis2-devs@lists.launchpad.net
>> >> > Unsubscribe : https://launchpad.net/~dhis2-devs
>> >> > More help : https://help.launchpad.net/ListHelp
>> >> >
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> --
>> >> Jason P. Pickering
>> >> email: jason.p.pickering@gmail.com
>> >> tel:+260968395190
>> >
>> >
>> >
>> >
>> > _______________________________________________
>> > Mailing list: https://launchpad.net/~dhis2-devs
>> > Post to : dhis2-devs@lists.launchpad.net
>> > Unsubscribe : https://launchpad.net/~dhis2-devs
>> > More help : https://help.launchpad.net/ListHelp
>> >
>> >
>
>
>
>

11

Good!

Hope will not be a problem said like that.

Anyway we need people using it. It is not a doubt. We know that on India there are MySQL with more than 5GB.

For vaccum yes postgres guys have daemonized the vaccum data clean. You are right.

Caveman

···

2010/7/12 Lars Helge Øverland larshelge@gmail.com

2010/7/12 Orvalho Augusto orvaquim@gmail.com:

Thanks!!

That of restoring a database of huge size is not everyday you do.

Well that depends :slight_smile:

I need people with databases of Postgres with sizes above 5GB. And tell us

how their postgres behave, specifically tell us how they address the vaccum

data thing.

Vacuum and analyze jobs are run as a daemon by default as far as I know.

From the postgres docs:

Maximum Database Size Unlimited

Maximum Table Size 32 TB

Maximum Row Size 1.6 TB

Maximum Field Size 1 GB

Maximum Rows per Table Unlimited

Maximum Columns per Table 250 - 1600 depending on column types

Maximum Indexes per Table Unlimited

So 5 GB should not be a problem…

Please is not a war.

Definitely not :slight_smile:

Caveman

2010/7/12 Lars Helge Øverland larshelge@gmail.com

Hi Caveman, we are definitely not dropping mysql support, its just

that most of the devs are using postgres so a function not being

mysql-compliant might sneak in from time to time. I tested most of the

application on mysql recently without trouble.

A common problem with aggregation earlier was the way the orgunit

hierarchy was represented and stored, this is changed in 2.0.5

snapshot and is way less error-prone now. So please build yourself a

war-file from trunk and test with that.

If that doesn’t work please send me a database dump and I will have a

look. (On a sidenote a main reason for choosing postgres is the way

dumps are restored - mysql uses single regular sql insert statements

which might take days for very large databases while postgres uses

more “native” copy statements which never takes more than minutes).

cheers, Lars

2010/7/12 Orvalho Augusto orvaquim@gmail.com:

I need to have a repo/table that aggregates data as I asked before. I

did

not find it on docs.

Someone told me that can happens if a generate those resource tables. I

try

to generate those tables and I get stuck on that error I reported.

Good for me if I do not need to generate such tables and ask what I

should

do?

Caveman

2010/7/12 Lars Helge Øverland larshelge@gmail.com

Generating any of the resource tables don’t affect the aggregation /

report tables…

On Sun, Jul 11, 2010 at 5:36 PM, Orvalho Augusto orvaquim@gmail.com > > >> >> wrote:

Thanks!!!

Caveman

On Sun, Jul 11, 2010 at 5:03 PM, Jason Pickering > > >> >> > jason.p.pickering@gmail.com wrote:

A very common mistake is not having generated the resource tables,

namely “orgunitstructure”. Consult the documentation for more

details

on how to generate these tables.

Regards,

Jason

On Sun, Jul 11, 2010 at 4:27 PM, Orvalho Augusto > > >> >> >> orvaquim@gmail.com > > >> >> >> wrote:

I have a dataelement collected at a Health Centre level (eg:

ats_posfem)

in

one province and only in some particular districts. In

unpredictible

future

at National level we will be collecting on every health centre.

Now, my ignorance: How can I obtain the provincial total on Report

Table? I

have been battled with DHIS so far and nothing.

Thanks guys

Caveman

Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

–

–

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190

Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

12

Hi Lars,
Well, yes and no. If the resource tables (namely orgunitstructure) is
not generated, it would not be possible to assign any orgunits to a
given data mart/report table? Likewise, if the orgunit levels have not
been defined, it is also not possible to assign any orgunits to a
given report table/datamart, or perhaps I am just hallucinating ? I
have run into this problem several times actually when creating new
databases. Ideally, the orgunitstructure table should be regenerated
when there have been changes to the orgunit hierarchy and users should
be warned that it is not possible to create a report table/datamart
without first defining orgunit level names.

Regards,
Jason

···

2010/7/12 Lars Helge Øverland <larshelge@gmail.com>:

Generating any of the resource tables don't affect the aggregation /
report tables...

On Sun, Jul 11, 2010 at 5:36 PM, Orvalho Augusto <orvaquim@gmail.com> wrote:

Thanks!!!

Caveman

On Sun, Jul 11, 2010 at 5:03 PM, Jason Pickering >> <jason.p.pickering@gmail.com> wrote:

A very common mistake is not having generated the resource tables,
namely "orgunitstructure". Consult the documentation for more details
on how to generate these tables.

Regards,
Jason

On Sun, Jul 11, 2010 at 4:27 PM, Orvalho Augusto <orvaquim@gmail.com> >>> wrote:
> I have a dataelement collected at a Health Centre level (eg: ats_posfem)
> in
> one province and only in some particular districts. In unpredictible
> future
> at National level we will be collecting on every health centre.
>
> Now, my ignorance: How can I obtain the provincial total on Report
> Table? I
> have been battled with DHIS so far and nothing.
>
> Thanks guys
> Caveman
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~dhis2-devs
> Post to : dhis2-devs@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~dhis2-devs
> More help : https://help.launchpad.net/ListHelp
>
>

--
--
Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+260968395190

_______________________________________________
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help : https://help.launchpad.net/ListHelp

--
--
Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+260968395190

13

I recompiled DHIS2.05 yesterday with help of a magician Jason yesterday. And the first thing I tried was this aggregation thing.

So I started redoing the orgunitstructure. It gave the same error.

While it was running I used the

show full processlist
to find out that it is a verry long insert to be run at once. I have 1664 orgunits. So it sounds like indexing trouble.

As there is a faith and “antimysql fight” I am moving my data to postgres. I have to do it manually.

I have to do it manually because DHIS does not keep de ids fixed (organisationunid, dataelementid etc). That affects our mapping data for data importation from other systems. So if someone could suggest a good strategy is welcomed.

Thanks for your patience guys
Caveman

···

2010/7/12 Jason Pickering jason.p.pickering@gmail.com

Hi Lars,

Well, yes and no. If the resource tables (namely orgunitstructure) is

not generated, it would not be possible to assign any orgunits to a

given data mart/report table? Likewise, if the orgunit levels have not

been defined, it is also not possible to assign any orgunits to a

given report table/datamart, or perhaps I am just hallucinating ? I

have run into this problem several times actually when creating new

databases. Ideally, the orgunitstructure table should be regenerated

when there have been changes to the orgunit hierarchy and users should

be warned that it is not possible to create a report table/datamart

without first defining orgunit level names.

Regards,

Jason

2010/7/12 Lars Helge Øverland larshelge@gmail.com:

Generating any of the resource tables don’t affect the aggregation /

report tables…

On Sun, Jul 11, 2010 at 5:36 PM, Orvalho Augusto orvaquim@gmail.com wrote:

Thanks!!!

Caveman

On Sun, Jul 11, 2010 at 5:03 PM, Jason Pickering > > >> jason.p.pickering@gmail.com wrote:

A very common mistake is not having generated the resource tables,

namely “orgunitstructure”. Consult the documentation for more details

on how to generate these tables.

Regards,

Jason

On Sun, Jul 11, 2010 at 4:27 PM, Orvalho Augusto orvaquim@gmail.com > > >>> wrote:

I have a dataelement collected at a Health Centre level (eg: ats_posfem)

in

one province and only in some particular districts. In unpredictible

future

at National level we will be collecting on every health centre.

Now, my ignorance: How can I obtain the provincial total on Report

Table? I

have been battled with DHIS so far and nothing.

Thanks guys

Caveman

Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

–

–

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190

Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

–

–

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190

14

I recompiled DHIS2.05 yesterday with help of a magician Jason yesterday. And
the first thing I tried was this aggregation thing.

So I started redoing the orgunitstructure. It gave the same error.

While it was running I used the
show full processlist
to find out that it is a verry long insert to be run at once. I have 1664
orgunits. So it sounds like indexing trouble.

As there is a faith and "antimysql fight" I am moving my data to postgres. I
have to do it manually.

There is no mysql fight. Just more people using postgres I guess.

I have to do it manually because DHIS does not keep de ids fixed
(organisationunid, dataelementid etc). That affects our mapping data for
data importation from other systems. So if someone could suggest a good
strategy is welcomed.

How are you doing mapping data from other systems? If you tell me a
bit about these systems and the data they have maybe we can make a
plan :slight_smile:

Cheers
Bob

···

On 13 July 2010 07:17, Orvalho Augusto <orvaquim@gmail.com> wrote:

Thanks for your patience guys
Caveman

2010/7/12 Jason Pickering <jason.p.pickering@gmail.com>

Hi Lars,
Well, yes and no. If the resource tables (namely orgunitstructure) is
not generated, it would not be possible to assign any orgunits to a
given data mart/report table? Likewise, if the orgunit levels have not
been defined, it is also not possible to assign any orgunits to a
given report table/datamart, or perhaps I am just hallucinating ? I
have run into this problem several times actually when creating new
databases. Ideally, the orgunitstructure table should be regenerated
when there have been changes to the orgunit hierarchy and users should
be warned that it is not possible to create a report table/datamart
without first defining orgunit level names.

Regards,
Jason

2010/7/12 Lars Helge Øverland <larshelge@gmail.com>:
> Generating any of the resource tables don't affect the aggregation /
> report tables...
>
>
>
> On Sun, Jul 11, 2010 at 5:36 PM, Orvalho Augusto <orvaquim@gmail.com> >> > wrote:
>> Thanks!!!
>>
>> Caveman
>>
>>
>> On Sun, Jul 11, 2010 at 5:03 PM, Jason Pickering >> >> <jason.p.pickering@gmail.com> wrote:
>>>
>>> A very common mistake is not having generated the resource tables,
>>> namely "orgunitstructure". Consult the documentation for more details
>>> on how to generate these tables.
>>>
>>> Regards,
>>> Jason
>>>
>>>
>>> On Sun, Jul 11, 2010 at 4:27 PM, Orvalho Augusto <orvaquim@gmail.com> >> >>> wrote:
>>> > I have a dataelement collected at a Health Centre level (eg:
>>> > ats_posfem)
>>> > in
>>> > one province and only in some particular districts. In unpredictible
>>> > future
>>> > at National level we will be collecting on every health centre.
>>> >
>>> > Now, my ignorance: How can I obtain the provincial total on Report
>>> > Table? I
>>> > have been battled with DHIS so far and nothing.
>>> >
>>> > Thanks guys
>>> > Caveman
>>> >
>>> >
>>> > _______________________________________________
>>> > Mailing list: https://launchpad.net/~dhis2-devs
>>> > Post to : dhis2-devs@lists.launchpad.net
>>> > Unsubscribe : https://launchpad.net/~dhis2-devs
>>> > More help : https://help.launchpad.net/ListHelp
>>> >
>>> >
>>>
>>>
>>>
>>> --
>>> --
>>> Jason P. Pickering
>>> email: jason.p.pickering@gmail.com
>>> tel:+260968395190
>>
>>
>>
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~dhis2-devs
>> Post to : dhis2-devs@lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~dhis2-devs
>> More help : https://help.launchpad.net/ListHelp
>>
>>
>

--
--
Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+260968395190

_______________________________________________
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help : https://help.launchpad.net/ListHelp

15

Hi Lars,

Well, yes and no. If the resource tables (namely orgunitstructure) is

not generated, it would not be possible to assign any orgunits to a

given data mart/report table?

The orgunitstructure table is just a resource table intended to be used by 3rd party tools which is not used internally in the system (maybe for the databrowser after you and Hieu changed it?).

Likewise, if the orgunit levels have not

been defined, it is also not possible to assign any orgunits to a

given report table/datamart, or perhaps I am just hallucinating ?

That is true, but I wouldn’t call the orgunit levels a resource table per se, it has a belonging object in the model and is used internally (even if you very well might use it in query).

I

have run into this problem several times actually when creating new

databases. Ideally, the orgunitstructure table should be regenerated

when there have been changes to the orgunit hierarchy and users should

be warned that it is not possible to create a report table/datamart

without first defining orgunit level names.

Agree that we could have some sort of integrated “required setup guide” in the system…

···

2010/7/12 Jason Pickering jason.p.pickering@gmail.com

Regards,

Jason

2010/7/12 Lars Helge Øverland larshelge@gmail.com:

Generating any of the resource tables don’t affect the aggregation /

report tables…

On Sun, Jul 11, 2010 at 5:36 PM, Orvalho Augusto orvaquim@gmail.com wrote:

Thanks!!!

Caveman

On Sun, Jul 11, 2010 at 5:03 PM, Jason Pickering > > >> jason.p.pickering@gmail.com wrote:

A very common mistake is not having generated the resource tables,

namely “orgunitstructure”. Consult the documentation for more details

on how to generate these tables.

Regards,

Jason

On Sun, Jul 11, 2010 at 4:27 PM, Orvalho Augusto orvaquim@gmail.com > > >>> wrote:

I have a dataelement collected at a Health Centre level (eg: ats_posfem)

in

one province and only in some particular districts. In unpredictible

future

at National level we will be collecting on every health centre.

Now, my ignorance: How can I obtain the provincial total on Report

Table? I

have been battled with DHIS so far and nothing.

Thanks guys

Caveman

Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

–

–

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190

Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

–

–

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190

16

I recompiled DHIS2.05 yesterday with help of a magician Jason yesterday. And

the first thing I tried was this aggregation thing.

So I started redoing the orgunitstructure. It gave the same error.

While it was running I used the

show full processlist

to find out that it is a verry long insert to be run at once. I have 1664

orgunits. So it sounds like indexing trouble.

As there is a faith and “antimysql fight” I am moving my data to postgres. I

have to do it manually.

There is no mysql fight. Just more people using postgres I guess.

I have to do it manually because DHIS does not keep de ids fixed

(organisationunid, dataelementid etc). That affects our mapping data for

data importation from other systems. So if someone could suggest a good

strategy is welcomed.

Database internal ids should not really be used to map anything between databases… Also postgres is using a sequence meaning auto-generated values are globally unique, while auto-generated values in mysql starts at 1 for each table.

···

On Tue, Jul 13, 2010 at 11:55 AM, Bob Jolliffe bobjolliffe@gmail.com wrote:

On 13 July 2010 07:17, Orvalho Augusto orvaquim@gmail.com wrote:

How are you doing mapping data from other systems? If you tell me a

bit about these systems and the data they have maybe we can make a

plan :slight_smile:

Cheers

Bob

Thanks for your patience guys

Caveman

2010/7/12 Jason Pickering jason.p.pickering@gmail.com

Hi Lars,

Well, yes and no. If the resource tables (namely orgunitstructure) is

not generated, it would not be possible to assign any orgunits to a

given data mart/report table? Likewise, if the orgunit levels have not

been defined, it is also not possible to assign any orgunits to a

given report table/datamart, or perhaps I am just hallucinating ? I

have run into this problem several times actually when creating new

databases. Ideally, the orgunitstructure table should be regenerated

when there have been changes to the orgunit hierarchy and users should

be warned that it is not possible to create a report table/datamart

without first defining orgunit level names.

Regards,

Jason

2010/7/12 Lars Helge Øverland larshelge@gmail.com:

Generating any of the resource tables don’t affect the aggregation /

report tables…

On Sun, Jul 11, 2010 at 5:36 PM, Orvalho Augusto orvaquim@gmail.com > > >> > wrote:

Thanks!!!

Caveman

On Sun, Jul 11, 2010 at 5:03 PM, Jason Pickering > > >> >> jason.p.pickering@gmail.com wrote:

A very common mistake is not having generated the resource tables,

namely “orgunitstructure”. Consult the documentation for more details

on how to generate these tables.

Regards,

Jason

On Sun, Jul 11, 2010 at 4:27 PM, Orvalho Augusto orvaquim@gmail.com > > >> >>> wrote:

I have a dataelement collected at a Health Centre level (eg:

ats_posfem)

in

one province and only in some particular districts. In unpredictible

future

at National level we will be collecting on every health centre.

Now, my ignorance: How can I obtain the provincial total on Report

Table? I

have been battled with DHIS so far and nothing.

Thanks guys

Caveman

Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

–

–

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190

Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

–

–

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190

Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

17

Ok.

I recompiled DHIS2.05 yesterday with help of a magician Jason yesterday. And

the first thing I tried was this aggregation thing.

So I started redoing the orgunitstructure. It gave the same error.

While it was running I used the

show full processlist

to find out that it is a verry long insert to be run at once. I have 1664

orgunits. So it sounds like indexing trouble.

As there is a faith and “antimysql fight” I am moving my data to postgres. I

have to do it manually.

There is no mysql fight. Just more people using postgres I guess.

I have to do it manually because DHIS does not keep de ids fixed

(organisationunid, dataelementid etc). That affects our mapping data for

data importation from other systems. So if someone could suggest a good

strategy is welcomed.

How are you doing mapping data from other systems? If you tell me a

bit about these systems and the data they have maybe we can make a

plan :slight_smile:
Basically these systems are:
The dataset (the DHIS concept) is represented by a table (in EpiInfo 6.06, DBF or MSAccess) where each record should have a month (or week or quarter), year, province, district and health center columns - sometimes they place the name of the association or a special sector of the health center - then the other columns define a dataelement for DHIS.

So what we were doing is:

  1. Migrate the data from anywhere (EpiInfo, MSAccess and DBF) to MySQL.

  2. On MySQL we have tables where the columns names of those tables mapping to the dataelementid (which Lars recommend not be like this)

  3. There is a script on Python made to run routinely to process this data and insert the new one on DHIS 2 tables.
    When I say routinely, I mean each month or whenever I want it.

The objective were to use solely DHIS for report and one day for data entry.

Caveman

···

On Tue, Jul 13, 2010 at 11:55 AM, Bob Jolliffe bobjolliffe@gmail.com wrote:

On 13 July 2010 07:17, Orvalho Augusto orvaquim@gmail.com wrote:

Cheers

Bob

Thanks for your patience guys

Caveman

2010/7/12 Jason Pickering jason.p.pickering@gmail.com

Hi Lars,

Well, yes and no. If the resource tables (namely orgunitstructure) is

not generated, it would not be possible to assign any orgunits to a

given data mart/report table? Likewise, if the orgunit levels have not

been defined, it is also not possible to assign any orgunits to a

given report table/datamart, or perhaps I am just hallucinating ? I

have run into this problem several times actually when creating new

databases. Ideally, the orgunitstructure table should be regenerated

when there have been changes to the orgunit hierarchy and users should

be warned that it is not possible to create a report table/datamart

without first defining orgunit level names.

Regards,

Jason

2010/7/12 Lars Helge Øverland larshelge@gmail.com:

Generating any of the resource tables don’t affect the aggregation /

report tables…

On Sun, Jul 11, 2010 at 5:36 PM, Orvalho Augusto orvaquim@gmail.com > > >> > wrote:

Thanks!!!

Caveman

On Sun, Jul 11, 2010 at 5:03 PM, Jason Pickering > > >> >> jason.p.pickering@gmail.com wrote:

A very common mistake is not having generated the resource tables,

namely “orgunitstructure”. Consult the documentation for more details

on how to generate these tables.

Regards,

Jason

On Sun, Jul 11, 2010 at 4:27 PM, Orvalho Augusto orvaquim@gmail.com > > >> >>> wrote:

I have a dataelement collected at a Health Centre level (eg:

ats_posfem)

in

one province and only in some particular districts. In unpredictible

future

at National level we will be collecting on every health centre.

Now, my ignorance: How can I obtain the provincial total on Report

Table? I

have been battled with DHIS so far and nothing.

Thanks guys

Caveman

Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

–

–

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190

Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

–

–

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190

Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

18

I have changed the database (from MySQL to Postgres)… and used DHIS tools to export and import the data.

Well, those resource tables are being built in seconds on Postgres now. So I think there is a bug with MySQL. And I want to help to correct it.

One verry bad consequence of not importing the dataelementids is: The entering forms does not work no more. We have to redesign the forms after an import.

Caveman

···

On Tue, Jul 13, 2010 at 10:28 PM, Orvalho Augusto orvaquim@gmail.com wrote:

Ok.

On Tue, Jul 13, 2010 at 11:55 AM, Bob Jolliffe bobjolliffe@gmail.com wrote:

On 13 July 2010 07:17, Orvalho Augusto orvaquim@gmail.com wrote:

I recompiled DHIS2.05 yesterday with help of a magician Jason yesterday. And

the first thing I tried was this aggregation thing.

So I started redoing the orgunitstructure. It gave the same error.

While it was running I used the

show full processlist

to find out that it is a verry long insert to be run at once. I have 1664

orgunits. So it sounds like indexing trouble.

As there is a faith and “antimysql fight” I am moving my data to postgres. I

have to do it manually.

There is no mysql fight. Just more people using postgres I guess.

I have to do it manually because DHIS does not keep de ids fixed

(organisationunid, dataelementid etc). That affects our mapping data for

data importation from other systems. So if someone could suggest a good

strategy is welcomed.

How are you doing mapping data from other systems? If you tell me a

bit about these systems and the data they have maybe we can make a

plan :slight_smile:

Basically these systems are:
The dataset (the DHIS concept) is represented by a table (in EpiInfo 6.06, DBF or MSAccess) where each record should have a month (or week or quarter), year, province, district and health center columns - sometimes they place the name of the association or a special sector of the health center - then the other columns define a dataelement for DHIS.

So what we were doing is:

  1. Migrate the data from anywhere (EpiInfo, MSAccess and DBF) to MySQL.

  2. On MySQL we have tables where the columns names of those tables mapping to the dataelementid (which Lars recommend not be like this)

  3. There is a script on Python made to run routinely to process this data and insert the new one on DHIS 2 tables.
    When I say routinely, I mean each month or whenever I want it.

The objective were to use solely DHIS for report and one day for data entry.

Caveman

Cheers

Bob

Thanks for your patience guys

Caveman

2010/7/12 Jason Pickering jason.p.pickering@gmail.com

Hi Lars,

Well, yes and no. If the resource tables (namely orgunitstructure) is

not generated, it would not be possible to assign any orgunits to a

given data mart/report table? Likewise, if the orgunit levels have not

been defined, it is also not possible to assign any orgunits to a

given report table/datamart, or perhaps I am just hallucinating ? I

have run into this problem several times actually when creating new

databases. Ideally, the orgunitstructure table should be regenerated

when there have been changes to the orgunit hierarchy and users should

be warned that it is not possible to create a report table/datamart

without first defining orgunit level names.

Regards,

Jason

2010/7/12 Lars Helge Øverland larshelge@gmail.com:

Generating any of the resource tables don’t affect the aggregation /

report tables…

On Sun, Jul 11, 2010 at 5:36 PM, Orvalho Augusto orvaquim@gmail.com > > > > >> > wrote:

Thanks!!!

Caveman

On Sun, Jul 11, 2010 at 5:03 PM, Jason Pickering > > > > >> >> jason.p.pickering@gmail.com wrote:

A very common mistake is not having generated the resource tables,

namely “orgunitstructure”. Consult the documentation for more details

on how to generate these tables.

Regards,

Jason

On Sun, Jul 11, 2010 at 4:27 PM, Orvalho Augusto orvaquim@gmail.com > > > > >> >>> wrote:

I have a dataelement collected at a Health Centre level (eg:

ats_posfem)

in

one province and only in some particular districts. In unpredictible

future

at National level we will be collecting on every health centre.

Now, my ignorance: How can I obtain the provincial total on Report

Table? I

have been battled with DHIS so far and nothing.

Thanks guys

Caveman

Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

–

–

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190

Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

–

–

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190

Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

19

And even do those resource tables on Postgres it does not aggregate.

Caveman

···

On Wed, Jul 14, 2010 at 6:35 AM, Orvalho Augusto orvaquim@gmail.com wrote:

I have changed the database (from MySQL to Postgres)… and used DHIS tools to export and import the data.

Well, those resource tables are being built in seconds on Postgres now. So I think there is a bug with MySQL. And I want to help to correct it.

One verry bad consequence of not importing the dataelementids is: The entering forms does not work no more. We have to redesign the forms after an import.

Caveman

On Tue, Jul 13, 2010 at 10:28 PM, Orvalho Augusto orvaquim@gmail.com wrote:

Ok.

On Tue, Jul 13, 2010 at 11:55 AM, Bob Jolliffe bobjolliffe@gmail.com wrote:

On 13 July 2010 07:17, Orvalho Augusto orvaquim@gmail.com wrote:

I recompiled DHIS2.05 yesterday with help of a magician Jason yesterday. And

the first thing I tried was this aggregation thing.

So I started redoing the orgunitstructure. It gave the same error.

While it was running I used the

show full processlist

to find out that it is a verry long insert to be run at once. I have 1664

orgunits. So it sounds like indexing trouble.

As there is a faith and “antimysql fight” I am moving my data to postgres. I

have to do it manually.

There is no mysql fight. Just more people using postgres I guess.

I have to do it manually because DHIS does not keep de ids fixed

(organisationunid, dataelementid etc). That affects our mapping data for

data importation from other systems. So if someone could suggest a good

strategy is welcomed.

How are you doing mapping data from other systems? If you tell me a

bit about these systems and the data they have maybe we can make a

plan :slight_smile:

Basically these systems are:
The dataset (the DHIS concept) is represented by a table (in EpiInfo 6.06, DBF or MSAccess) where each record should have a month (or week or quarter), year, province, district and health center columns - sometimes they place the name of the association or a special sector of the health center - then the other columns define a dataelement for DHIS.

So what we were doing is:

  1. Migrate the data from anywhere (EpiInfo, MSAccess and DBF) to MySQL.

  2. On MySQL we have tables where the columns names of those tables mapping to the dataelementid (which Lars recommend not be like this)

  3. There is a script on Python made to run routinely to process this data and insert the new one on DHIS 2 tables.
    When I say routinely, I mean each month or whenever I want it.

The objective were to use solely DHIS for report and one day for data entry.

Caveman

Cheers

Bob

Thanks for your patience guys

Caveman

2010/7/12 Jason Pickering jason.p.pickering@gmail.com

Hi Lars,

Well, yes and no. If the resource tables (namely orgunitstructure) is

not generated, it would not be possible to assign any orgunits to a

given data mart/report table? Likewise, if the orgunit levels have not

been defined, it is also not possible to assign any orgunits to a

given report table/datamart, or perhaps I am just hallucinating ? I

have run into this problem several times actually when creating new

databases. Ideally, the orgunitstructure table should be regenerated

when there have been changes to the orgunit hierarchy and users should

be warned that it is not possible to create a report table/datamart

without first defining orgunit level names.

Regards,

Jason

2010/7/12 Lars Helge Øverland larshelge@gmail.com:

Generating any of the resource tables don’t affect the aggregation /

report tables…

On Sun, Jul 11, 2010 at 5:36 PM, Orvalho Augusto orvaquim@gmail.com > > > > > > >> > wrote:

Thanks!!!

Caveman

On Sun, Jul 11, 2010 at 5:03 PM, Jason Pickering > > > > > > >> >> jason.p.pickering@gmail.com wrote:

A very common mistake is not having generated the resource tables,

namely “orgunitstructure”. Consult the documentation for more details

on how to generate these tables.

Regards,

Jason

On Sun, Jul 11, 2010 at 4:27 PM, Orvalho Augusto orvaquim@gmail.com > > > > > > >> >>> wrote:

I have a dataelement collected at a Health Centre level (eg:

ats_posfem)

in

one province and only in some particular districts. In unpredictible

future

at National level we will be collecting on every health centre.

Now, my ignorance: How can I obtain the provincial total on Report

Table? I

have been battled with DHIS so far and nothing.

Thanks guys

Caveman

Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

–

–

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190

Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

–

–

Jason P. Pickering

email: jason.p.pickering@gmail.com

tel:+260968395190

Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

20

Hi Caveman,

Can you please post a stack trace of your application (use
pastebin.com) from startup until you start the datamart/report table
process?

Do you have any external views that are linked to the report table you
are attempting to aggregate?

Regards,
Jason

···

On Wed, Jul 14, 2010 at 8:57 AM, Orvalho Augusto <orvaquim@gmail.com> wrote:

And even do those resource tables on Postgres it does not aggregate.

Caveman

On Wed, Jul 14, 2010 at 6:35 AM, Orvalho Augusto <orvaquim@gmail.com> wrote:

I have changed the database (from MySQL to Postgres)... and used DHIS
tools to export and import the data.

Well, those resource tables are being built in seconds on Postgres now. So
I think there is a bug with MySQL. And I want to help to correct it.

One verry bad consequence of not importing the dataelementids is: The
entering forms does not work no more. We have to redesign the forms after an
import.

Caveman

On Tue, Jul 13, 2010 at 10:28 PM, Orvalho Augusto <orvaquim@gmail.com> >> wrote:

Ok.

On Tue, Jul 13, 2010 at 11:55 AM, Bob Jolliffe <bobjolliffe@gmail.com> >>> wrote:

On 13 July 2010 07:17, Orvalho Augusto <orvaquim@gmail.com> wrote:
> I recompiled DHIS2.05 yesterday with help of a magician Jason
> yesterday. And
> the first thing I tried was this aggregation thing.
>
> So I started redoing the orgunitstructure. It gave the same error.
>
> While it was running I used the
> show full processlist
> to find out that it is a verry long insert to be run at once. I have
> 1664
> orgunits. So it sounds like indexing trouble.
>
> As there is a faith and "antimysql fight" I am moving my data to
> postgres. I
> have to do it manually.

There is no mysql fight. Just more people using postgres I guess.

>
> I have to do it manually because DHIS does not keep de ids fixed
> (organisationunid, dataelementid etc). That affects our mapping data
> for
> data importation from other systems. So if someone could suggest a
> good
> strategy is welcomed.

How are you doing mapping data from other systems? If you tell me a
bit about these systems and the data they have maybe we can make a
plan :slight_smile:

Basically these systems are:
The dataset (the DHIS concept) is represented by a table (in EpiInfo
6.06, DBF or MSAccess) where each record should have a month (or week or
quarter), year, province, district and health center columns - sometimes
they place the name of the association or a special sector of the health
center - then the other columns define a dataelement for DHIS.

So what we were doing is:
1. Migrate the data from anywhere (EpiInfo, MSAccess and DBF) to MySQL.

2. On MySQL we have tables where the columns names of those tables
mapping to the dataelementid (which Lars recommend not be like this)

3. There is a script on Python made to run routinely to process this data
and insert the new one on DHIS 2 tables.
When I say routinely, I mean each month or whenever I want it.

The objective were to use solely DHIS for report and one day for data
entry.

Caveman

Cheers
Bob

>
> Thanks for your patience guys
> Caveman
>
>
>
>
>
> 2010/7/12 Jason Pickering <jason.p.pickering@gmail.com>
>>
>> Hi Lars,
>> Well, yes and no. If the resource tables (namely orgunitstructure) is
>> not generated, it would not be possible to assign any orgunits to a
>> given data mart/report table? Likewise, if the orgunit levels have
>> not
>> been defined, it is also not possible to assign any orgunits to a
>> given report table/datamart, or perhaps I am just hallucinating ? I
>> have run into this problem several times actually when creating new
>> databases. Ideally, the orgunitstructure table should be regenerated
>> when there have been changes to the orgunit hierarchy and users
>> should
>> be warned that it is not possible to create a report table/datamart
>> without first defining orgunit level names.
>>
>> Regards,
>> Jason
>>
>>
>> 2010/7/12 Lars Helge Øverland <larshelge@gmail.com>:
>> > Generating any of the resource tables don't affect the aggregation
>> > /
>> > report tables...
>> >
>> >
>> >
>> > On Sun, Jul 11, 2010 at 5:36 PM, Orvalho Augusto >>>> >> > <orvaquim@gmail.com> >>>> >> > wrote:
>> >> Thanks!!!
>> >>
>> >> Caveman
>> >>
>> >>
>> >> On Sun, Jul 11, 2010 at 5:03 PM, Jason Pickering >>>> >> >> <jason.p.pickering@gmail.com> wrote:
>> >>>
>> >>> A very common mistake is not having generated the resource
>> >>> tables,
>> >>> namely "orgunitstructure". Consult the documentation for more
>> >>> details
>> >>> on how to generate these tables.
>> >>>
>> >>> Regards,
>> >>> Jason
>> >>>
>> >>>
>> >>> On Sun, Jul 11, 2010 at 4:27 PM, Orvalho Augusto >>>> >> >>> <orvaquim@gmail.com> >>>> >> >>> wrote:
>> >>> > I have a dataelement collected at a Health Centre level (eg:
>> >>> > ats_posfem)
>> >>> > in
>> >>> > one province and only in some particular districts. In
>> >>> > unpredictible
>> >>> > future
>> >>> > at National level we will be collecting on every health centre.
>> >>> >
>> >>> > Now, my ignorance: How can I obtain the provincial total on
>> >>> > Report
>> >>> > Table? I
>> >>> > have been battled with DHIS so far and nothing.
>> >>> >
>> >>> > Thanks guys
>> >>> > Caveman
>> >>> >
>> >>> >
>> >>> > _______________________________________________
>> >>> > Mailing list: https://launchpad.net/~dhis2-devs
>> >>> > Post to : dhis2-devs@lists.launchpad.net
>> >>> > Unsubscribe : https://launchpad.net/~dhis2-devs
>> >>> > More help : https://help.launchpad.net/ListHelp
>> >>> >
>> >>> >
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> --
>> >>> Jason P. Pickering
>> >>> email: jason.p.pickering@gmail.com
>> >>> tel:+260968395190
>> >>
>> >>
>> >>
>> >>
>> >> _______________________________________________
>> >> Mailing list: https://launchpad.net/~dhis2-devs
>> >> Post to : dhis2-devs@lists.launchpad.net
>> >> Unsubscribe : https://launchpad.net/~dhis2-devs
>> >> More help : https://help.launchpad.net/ListHelp
>> >>
>> >>
>> >
>>
>>
>>
>> --
>> --
>> Jason P. Pickering
>> email: jason.p.pickering@gmail.com
>> tel:+260968395190
>
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~dhis2-devs
> Post to : dhis2-devs@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~dhis2-devs
> More help : https://help.launchpad.net/ListHelp
>
>

--
--
Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+260968395190