Upgrade from 2.10 to 2.12 failing

Jason,
I had done those procedures before and I had not experience such issues. What I think should be done is to stop the tomcat, copy route file somewhere else and delete it. Then put the new war file and restart tomcat.

···

Sent from my BlackBerry® smartphone from Airtel Sierra Leone

-----Original Message-----
From: Jason Pickering <jason.p.pickering@gmail.com>
Date: Tue, 24 Sep 2013 10:45:10
To: <gerald.81@hotmail.com>
Cc: <dhis2-devs@lists.launchpad.net>
Subject: Re: [Dhis2-devs] Upgrade from 2.10 to 2.12 failing

Hi Gerald,

There is no procedure for this really other than replacing the WAR file. When you move from 2.10 to 2.12, you simply replace the old WAR file with the new one, and the application is supposed to ugprade itself automatically. However in this case, it does not , which indicates a problem in the startup routines and/or database structure.

Regards,
Jason

On Tue, Sep 24, 2013 at 12:35 PM, Gerald Thomas <gerald.81@hotmail.com <mailto:gerald.81@hotmail.com> > wrote:
You said, you are stuck on step 4/10 which means you have means you have an upgrade procedure installation file. That is the file I want you to share with me.

Sent from my BlackBerry® smartphone from Airtel Sierra Leone

-----Original Message-----
From: Jason Pickering <jason.p.pickering@gmail.com <mailto:jason.p.pickering@gmail.com> >

Date: Tue, 24 Sep 2013 10:32:40
To: <gerald.81@hotmail.com <mailto:gerald.81@hotmail.com> >
Subject: Re: [Dhis2-devs] Upgrade from 2.10 to 2.12 failing

Hi Gerald,
What is a steps file?

Regards,
Jason

On Tue, Sep 24, 2013 at 12:31 PM, Gerald Thomas <gerald.81@hotmail.com <mailto:gerald.81@hotmail.com> <mailto:gerald.81@hotmail.com <mailto:gerald.81@hotmail.com> > > wrote:
Please can you share the steps file you are using with me. Thanks in advance.
Sent from my BlackBerry® smartphone from Airtel Sierra Leone

-----Original Message-----

From: Jason Pickering <jason.p.pickering@gmail.com <mailto:jason.p.pickering@gmail.com> <mailto:jason.p.pickering@gmail.com <mailto:jason.p.pickering@gmail.com> > >
Date: Tue, 24 Sep 2013 10:26:56

To: <dhis2-devs@lists.launchpad.net <mailto:dhis2-devs@lists.launchpad.net> <mailto:dhis2-devs@lists.launchpad.net <mailto:dhis2-devs@lists.launchpad.net> > >
Subject: [Dhis2-devs] Upgrade from 2.10 to 2.12 failing

Using the latest 2.12 build from Jenkins.

Looks like something is failing on the startup, but not sure what?

This DB runs fine on 2.10. The first time around, a bunch of timestamps were updated. I killed the server, restarted, but still getting stuck at step 4 /10.

Any tips?

Regards,
Jason

* INFO 2013-09-24 11:21:23,080 Found the following message senders: [org.hisp.dhis.message.EmailMessageSender@17a220d7, org.hisp.dhis.sms.outbound.SmsSender@151a0f60] (DefaultMessageService.java [main])
* INFO 2013-09-24 11:21:39,093 Camel context started (DHIS2LifecycleStrategy.java [main])
* INFO 2013-09-24 11:21:39,094 Searching for routes in directory: /home/testinstance/dhis_home/routes (DHIS2LifecycleStrategy.java [main])
* INFO 2013-09-24 11:21:40,242 Executing startup routine [1 of 10, runlevel 1]: TableAlteror (DefaultStartupRoutineExecutor.java [main])
* INFO 2013-09-24 11:21:46,749 Tables updated (TableAlteror.java [main])
* INFO 2013-09-24 11:21:46,750 Executing startup routine [2 of 10, runlevel 1]: MapViewUpgrader (DefaultStartupRoutineExecutor.java [main])
* INFO 2013-09-24 11:21:47,012 Executing startup routine [3 of 10, runlevel 2]: IdentityPopulator (DefaultStartupRoutineExecutor.java [main])
Sep 24, 2013 11:21:48 AM org.apache.catalina.core.StandardContext start
SEVERE: Error listenerStart
Sep 24, 2013 11:21:48 AM org.apache.catalina.core.StandardContext start
SEVERE: Context [/testinstance] startup failed due to previous errors
* INFO 2013-09-24 11:21:48,017 De-registering jdbc driver: org.h2.Driver@65b8b5cd (StartupListener.java [main])
* INFO 2013-09-24 11:21:48,018 De-registering jdbc driver: org.hsqldb.jdbc.JDBCDriver@2830803a (StartupListener.java [main])
* INFO 2013-09-24 11:21:48,018 De-registering jdbc driver: com.mysql.jdbc.Driver@1ccdf3c2 (StartupListener.java [main])
* INFO 2013-09-24 11:21:48,018 De-registering jdbc driver: org.postgresql.Driver@2f1261b1 (StartupListener.java [main])
* INFO 2013-09-24 11:21:48,186 Camel context stopped (DHIS2LifecycleStrategy.java [main])
Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: A web application appears to have started a thread named [Resource Destroyer in BasicResourcePool.close()] but has failed to stop it. This is very likely to create a memory leak.
Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Transactional resources]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.
Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Transaction synchronizations]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.
Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Prototype beans currently in creation]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.
Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: A web application created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@9db900f]) and a value of type [org.springframework.security.core.context.SecurityContextImpl] (value [org.springframework.security.core.context.SecurityContextImpl@ffffffff: Null authentication]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.
Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Current transaction name]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.
Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Current aspect-driven transaction]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.
Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Prototype beans currently in creation]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.
Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Locale context]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.
Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Current transaction read-only status]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.
Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Current transaction isolation level]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.
Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Actual transaction active]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.
Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: A web application created a ThreadLocal with key of type [null] (value [com.sun.xml.bind.v2.ClassFactory$1@642bf5bf]) and a value of type [java.util.WeakHashMap] (value [{class org.apache.camel.spring.CamelProducerTemplateFactoryBean=java.lang.ref.WeakReference@32b7e05f, class org.apache.camel.model.PackageScanDefinition=java.lang.ref.WeakReference@45eb96fc, class javax.xml.bind.annotation.adapters.CollapsedStringAdapter=java.lang.ref.WeakReference@6e059e40, class java.util.ArrayList=java.lang.ref.WeakReference@32d8b42e, class org.apache.camel.spring.CamelContextFactoryBean=java.lang.ref.WeakReference@311d5216}]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.
Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedInheritableThreadLocal] (value [Locale context]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.
Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedInheritableThreadLocal] (value [Locale context]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.
Sep 24, 2013 11:21:49 AM org.apache.coyote.http11.Http11AprProtocol start
INFO: Starting Coyote HTTP/1.1 on http-8082
Sep 24, 2013 11:21:49 AM org.apache.catalina.startup.Catalina start
INFO: Server startup in 77181 ms

Hi Gerald,

I have done all of this already. Sorry, my procedure was not really detailed enough (kill Tomcat, remove WAR, remove directory in webapps, download WAR, restart Tomcat)

It is likely a structural problem or metadata problem with my particular database. On other databases, it works fine. There is just something peculiar about this database, which the startup routines are not handling properly.

Regards,

Jason

···

On Tue, Sep 24, 2013 at 12:50 PM, Gerald Thomas gerald.81@hotmail.com wrote:

Jason,

I had done those procedures before and I had not experience such issues. What I think should be done is to stop the tomcat, copy route file somewhere else and delete it. Then put the new war file and restart tomcat.
Sent from my BlackBerry® smartphone from Airtel Sierra Leone

-----Original Message-----

From: Jason Pickering jason.p.pickering@gmail.com

Date: Tue, 24 Sep 2013 10:45:10

To: gerald.81@hotmail.com

Cc: dhis2-devs@lists.launchpad.net

Subject: Re: [Dhis2-devs] Upgrade from 2.10 to 2.12 failing

Hi Gerald,

There is no procedure for this really other than replacing the WAR file. When you move from 2.10 to 2.12, you simply replace the old WAR file with the new one, and the application is supposed to ugprade itself automatically. However in this case, it does not , which indicates a problem in the startup routines and/or database structure.

Regards,

Jason

On Tue, Sep 24, 2013 at 12:35 PM, Gerald Thomas <gerald.81@hotmail.com mailto:gerald.81@hotmail.com > wrote:

You said, you are stuck on step 4/10 which means you have means you have an upgrade procedure installation file. That is the file I want you to share with me.

Sent from my BlackBerry® smartphone from Airtel Sierra Leone

-----Original Message-----

From: Jason Pickering <jason.p.pickering@gmail.com mailto:jason.p.pickering@gmail.com >

Date: Tue, 24 Sep 2013 10:32:40

To: <gerald.81@hotmail.com mailto:gerald.81@hotmail.com >
Subject: Re: [Dhis2-devs] Upgrade from 2.10 to 2.12 failing

Hi Gerald,

What is a steps file?

Regards,

Jason

On Tue, Sep 24, 2013 at 12:31 PM, Gerald Thomas <gerald.81@hotmail.com mailto:gerald.81@hotmail.com <mailto:gerald.81@hotmail.com mailto:gerald.81@hotmail.com > > wrote:

Please can you share the steps file you are using with me. Thanks in advance.

Sent from my BlackBerry® smartphone from Airtel Sierra Leone

-----Original Message-----

From: Jason Pickering <jason.p.pickering@gmail.com mailto:jason.p.pickering@gmail.com <mailto:jason.p.pickering@gmail.com mailto:jason.p.pickering@gmail.com > >
Date: Tue, 24 Sep 2013 10:26:56

To: <dhis2-devs@lists.launchpad.net mailto:dhis2-devs@lists.launchpad.net <mailto:dhis2-devs@lists.launchpad.net mailto:dhis2-devs@lists.launchpad.net > >
Subject: [Dhis2-devs] Upgrade from 2.10 to 2.12 failing

Using the latest 2.12 build from Jenkins.

Looks like something is failing on the startup, but not sure what?

This DB runs fine on 2.10. The first time around, a bunch of timestamps were updated. I killed the server, restarted, but still getting stuck at step 4 /10.

Any tips?

Regards,

Jason

  • INFO 2013-09-24 11:21:23,080 Found the following message senders: [org.hisp.dhis.message.EmailMessageSender@17a220d7, org.hisp.dhis.sms.outbound.SmsSender@151a0f60] (DefaultMessageService.java [main])

  • INFO 2013-09-24 11:21:39,093 Camel context started (DHIS2LifecycleStrategy.java [main])

  • INFO 2013-09-24 11:21:39,094 Searching for routes in directory: /home/testinstance/dhis_home/routes (DHIS2LifecycleStrategy.java [main])

  • INFO 2013-09-24 11:21:40,242 Executing startup routine [1 of 10, runlevel 1]: TableAlteror (DefaultStartupRoutineExecutor.java [main])

  • INFO 2013-09-24 11:21:46,749 Tables updated (TableAlteror.java [main])

  • INFO 2013-09-24 11:21:46,750 Executing startup routine [2 of 10, runlevel 1]: MapViewUpgrader (DefaultStartupRoutineExecutor.java [main])

  • INFO 2013-09-24 11:21:47,012 Executing startup routine [3 of 10, runlevel 2]: IdentityPopulator (DefaultStartupRoutineExecutor.java [main])

Sep 24, 2013 11:21:48 AM org.apache.catalina.core.StandardContext start

SEVERE: Error listenerStart

Sep 24, 2013 11:21:48 AM org.apache.catalina.core.StandardContext start

SEVERE: Context [/testinstance] startup failed due to previous errors

  • INFO 2013-09-24 11:21:48,017 De-registering jdbc driver: org.h2.Driver@65b8b5cd (StartupListener.java [main])

  • INFO 2013-09-24 11:21:48,018 De-registering jdbc driver: org.hsqldb.jdbc.JDBCDriver@2830803a (StartupListener.java [main])

  • INFO 2013-09-24 11:21:48,018 De-registering jdbc driver: com.mysql.jdbc.Driver@1ccdf3c2 (StartupListener.java [main])

  • INFO 2013-09-24 11:21:48,018 De-registering jdbc driver: org.postgresql.Driver@2f1261b1 (StartupListener.java [main])

  • INFO 2013-09-24 11:21:48,186 Camel context stopped (DHIS2LifecycleStrategy.java [main])

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads

SEVERE: A web application appears to have started a thread named [Resource Destroyer in BasicResourcePool.close()] but has failed to stop it. This is very likely to create a memory leak.

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Transactional resources]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Transaction synchronizations]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Prototype beans currently in creation]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

SEVERE: A web application created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@9db900f]) and a value of type [org.springframework.security.core.context.SecurityContextImpl] (value [org.springframework.security.core.context.SecurityContextImpl@ffffffff: Null authentication]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Current transaction name]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Current aspect-driven transaction]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Prototype beans currently in creation]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Locale context]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Current transaction read-only status]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Current transaction isolation level]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Actual transaction active]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

SEVERE: A web application created a ThreadLocal with key of type [null] (value [com.sun.xml.bind.v2.ClassFactory$1@642bf5bf]) and a value of type [java.util.WeakHashMap] (value [{class org.apache.camel.spring.CamelProducerTemplateFactoryBean=java.lang.ref.WeakReference@32b7e05f, class org.apache.camel.model.PackageScanDefinition=java.lang.ref.WeakReference@45eb96fc, class javax.xml.bind.annotation.adapters.CollapsedStringAdapter=java.lang.ref.WeakReference@6e059e40, class java.util.ArrayList=java.lang.ref.WeakReference@32d8b42e, class org.apache.camel.spring.CamelContextFactoryBean=java.lang.ref.WeakReference@311d5216}]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedInheritableThreadLocal] (value [Locale context]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedInheritableThreadLocal] (value [Locale context]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Sep 24, 2013 11:21:49 AM org.apache.coyote.http11.Http11AprProtocol start

INFO: Starting Coyote HTTP/1.1 on http-8082

Sep 24, 2013 11:21:49 AM org.apache.catalina.startup.Catalina start

INFO: Server startup in 77181 ms

Hi Jason, are you sure you don’t need to do 2.11 first?

···

On Tue, Sep 24, 2013 at 12:50 PM, Gerald Thomas gerald.81@hotmail.com wrote:

Jason,

I had done those procedures before and I had not experience such issues. What I think should be done is to stop the tomcat, copy route file somewhere else and delete it. Then put the new war file and restart tomcat.
Sent from my BlackBerry® smartphone from Airtel Sierra Leone

-----Original Message-----

From: Jason Pickering jason.p.pickering@gmail.com

Date: Tue, 24 Sep 2013 10:45:10

To: gerald.81@hotmail.com

Cc: dhis2-devs@lists.launchpad.net

Subject: Re: [Dhis2-devs] Upgrade from 2.10 to 2.12 failing

Hi Gerald,

There is no procedure for this really other than replacing the WAR file. When you move from 2.10 to 2.12, you simply replace the old WAR file with the new one, and the application is supposed to ugprade itself automatically. However in this case, it does not , which indicates a problem in the startup routines and/or database structure.

Regards,

Jason

On Tue, Sep 24, 2013 at 12:35 PM, Gerald Thomas <gerald.81@hotmail.com mailto:gerald.81@hotmail.com > wrote:

You said, you are stuck on step 4/10 which means you have means you have an upgrade procedure installation file. That is the file I want you to share with me.

Sent from my BlackBerry® smartphone from Airtel Sierra Leone

-----Original Message-----

From: Jason Pickering <jason.p.pickering@gmail.com mailto:jason.p.pickering@gmail.com >

Date: Tue, 24 Sep 2013 10:32:40

To: <gerald.81@hotmail.com mailto:gerald.81@hotmail.com >
Subject: Re: [Dhis2-devs] Upgrade from 2.10 to 2.12 failing

Hi Gerald,

What is a steps file?

Regards,

Jason

On Tue, Sep 24, 2013 at 12:31 PM, Gerald Thomas <gerald.81@hotmail.com mailto:gerald.81@hotmail.com <mailto:gerald.81@hotmail.com mailto:gerald.81@hotmail.com > > wrote:

Please can you share the steps file you are using with me. Thanks in advance.

Sent from my BlackBerry® smartphone from Airtel Sierra Leone

-----Original Message-----

From: Jason Pickering <jason.p.pickering@gmail.com mailto:jason.p.pickering@gmail.com <mailto:jason.p.pickering@gmail.com mailto:jason.p.pickering@gmail.com > >
Date: Tue, 24 Sep 2013 10:26:56

To: <dhis2-devs@lists.launchpad.net mailto:dhis2-devs@lists.launchpad.net <mailto:dhis2-devs@lists.launchpad.net mailto:dhis2-devs@lists.launchpad.net > >
Subject: [Dhis2-devs] Upgrade from 2.10 to 2.12 failing

Using the latest 2.12 build from Jenkins.

Looks like something is failing on the startup, but not sure what?

This DB runs fine on 2.10. The first time around, a bunch of timestamps were updated. I killed the server, restarted, but still getting stuck at step 4 /10.

Any tips?

Regards,

Jason

  • INFO 2013-09-24 11:21:23,080 Found the following message senders: [org.hisp.dhis.message.EmailMessageSender@17a220d7, org.hisp.dhis.sms.outbound.SmsSender@151a0f60] (DefaultMessageService.java [main])

  • INFO 2013-09-24 11:21:39,093 Camel context started (DHIS2LifecycleStrategy.java [main])

  • INFO 2013-09-24 11:21:39,094 Searching for routes in directory: /home/testinstance/dhis_home/routes (DHIS2LifecycleStrategy.java [main])

  • INFO 2013-09-24 11:21:40,242 Executing startup routine [1 of 10, runlevel 1]: TableAlteror (DefaultStartupRoutineExecutor.java [main])

  • INFO 2013-09-24 11:21:46,749 Tables updated (TableAlteror.java [main])

  • INFO 2013-09-24 11:21:46,750 Executing startup routine [2 of 10, runlevel 1]: MapViewUpgrader (DefaultStartupRoutineExecutor.java [main])

  • INFO 2013-09-24 11:21:47,012 Executing startup routine [3 of 10, runlevel 2]: IdentityPopulator (DefaultStartupRoutineExecutor.java [main])

Sep 24, 2013 11:21:48 AM org.apache.catalina.core.StandardContext start

SEVERE: Error listenerStart

Sep 24, 2013 11:21:48 AM org.apache.catalina.core.StandardContext start

SEVERE: Context [/testinstance] startup failed due to previous errors

  • INFO 2013-09-24 11:21:48,017 De-registering jdbc driver: org.h2.Driver@65b8b5cd (StartupListener.java [main])

  • INFO 2013-09-24 11:21:48,018 De-registering jdbc driver: org.hsqldb.jdbc.JDBCDriver@2830803a (StartupListener.java [main])

  • INFO 2013-09-24 11:21:48,018 De-registering jdbc driver: com.mysql.jdbc.Driver@1ccdf3c2 (StartupListener.java [main])

  • INFO 2013-09-24 11:21:48,018 De-registering jdbc driver: org.postgresql.Driver@2f1261b1 (StartupListener.java [main])

  • INFO 2013-09-24 11:21:48,186 Camel context stopped (DHIS2LifecycleStrategy.java [main])

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads

SEVERE: A web application appears to have started a thread named [Resource Destroyer in BasicResourcePool.close()] but has failed to stop it. This is very likely to create a memory leak.

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Transactional resources]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Transaction synchronizations]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Prototype beans currently in creation]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

SEVERE: A web application created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@9db900f]) and a value of type [org.springframework.security.core.context.SecurityContextImpl] (value [org.springframework.security.core.context.SecurityContextImpl@ffffffff: Null authentication]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Current transaction name]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Current aspect-driven transaction]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Prototype beans currently in creation]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Locale context]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Current transaction read-only status]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Current transaction isolation level]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Actual transaction active]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

SEVERE: A web application created a ThreadLocal with key of type [null] (value [com.sun.xml.bind.v2.ClassFactory$1@642bf5bf]) and a value of type [java.util.WeakHashMap] (value [{class org.apache.camel.spring.CamelProducerTemplateFactoryBean=java.lang.ref.WeakReference@32b7e05f, class org.apache.camel.model.PackageScanDefinition=java.lang.ref.WeakReference@45eb96fc, class javax.xml.bind.annotation.adapters.CollapsedStringAdapter=java.lang.ref.WeakReference@6e059e40, class java.util.ArrayList=java.lang.ref.WeakReference@32d8b42e, class org.apache.camel.spring.CamelContextFactoryBean=java.lang.ref.WeakReference@311d5216}]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedInheritableThreadLocal] (value [Locale context]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedInheritableThreadLocal] (value [Locale context]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Sep 24, 2013 11:21:49 AM org.apache.coyote.http11.Http11AprProtocol start

INFO: Starting Coyote HTTP/1.1 on http-8082

Sep 24, 2013 11:21:49 AM org.apache.catalina.startup.Catalina start

INFO: Server startup in 77181 ms

Hi Knut,

Yeah, I tried this as well, with the same result.

I tried going from 2.10 to 2.12. Rolled back to the original database (2.10) , and tried to go from 2.10 to 2.11 and then 2.12. 2.11 manages to start, but with a lot of errors like…

Failed to upgrade expression: ([26.13]*1)+([27.13]*1)+([31.13]*1)+([32.13]*1)+([33.13]*1)+([34.13]*1) (ExpressionUpgrader.java [main]).

I have seen this before on other databases, and it has not proven to be be fatal, so I suspect it is something else, but just not quite sure what, but might be part of the problem.

Regards,

Jason

···

On Tue, Sep 24, 2013 at 3:01 PM, Knut Staring knutst@gmail.com wrote:

Hi Jason, are you sure you don’t need to do 2.11 first?

Den 24. sep. 2013 19:01 skrev “Jason Pickering” jason.p.pickering@gmail.com følgende:

Hi Gerald,

I have done all of this already. Sorry, my procedure was not really detailed enough (kill Tomcat, remove WAR, remove directory in webapps, download WAR, restart Tomcat)

It is likely a structural problem or metadata problem with my particular database. On other databases, it works fine. There is just something peculiar about this database, which the startup routines are not handling properly.

Regards,

Jason


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

On Tue, Sep 24, 2013 at 12:50 PM, Gerald Thomas gerald.81@hotmail.com wrote:

Jason,

I had done those procedures before and I had not experience such issues. What I think should be done is to stop the tomcat, copy route file somewhere else and delete it. Then put the new war file and restart tomcat.
Sent from my BlackBerry® smartphone from Airtel Sierra Leone

-----Original Message-----

From: Jason Pickering jason.p.pickering@gmail.com

Date: Tue, 24 Sep 2013 10:45:10

To: gerald.81@hotmail.com

Cc: dhis2-devs@lists.launchpad.net

Subject: Re: [Dhis2-devs] Upgrade from 2.10 to 2.12 failing

Hi Gerald,

There is no procedure for this really other than replacing the WAR file. When you move from 2.10 to 2.12, you simply replace the old WAR file with the new one, and the application is supposed to ugprade itself automatically. However in this case, it does not , which indicates a problem in the startup routines and/or database structure.

Regards,

Jason

On Tue, Sep 24, 2013 at 12:35 PM, Gerald Thomas <gerald.81@hotmail.com mailto:gerald.81@hotmail.com > wrote:

You said, you are stuck on step 4/10 which means you have means you have an upgrade procedure installation file. That is the file I want you to share with me.

Sent from my BlackBerry® smartphone from Airtel Sierra Leone

-----Original Message-----

From: Jason Pickering <jason.p.pickering@gmail.com mailto:jason.p.pickering@gmail.com >

Date: Tue, 24 Sep 2013 10:32:40

To: <gerald.81@hotmail.com mailto:gerald.81@hotmail.com >
Subject: Re: [Dhis2-devs] Upgrade from 2.10 to 2.12 failing

Hi Gerald,

What is a steps file?

Regards,

Jason

On Tue, Sep 24, 2013 at 12:31 PM, Gerald Thomas <gerald.81@hotmail.com mailto:gerald.81@hotmail.com <mailto:gerald.81@hotmail.com mailto:gerald.81@hotmail.com > > wrote:

Please can you share the steps file you are using with me. Thanks in advance.

Sent from my BlackBerry® smartphone from Airtel Sierra Leone

-----Original Message-----

From: Jason Pickering <jason.p.pickering@gmail.com mailto:jason.p.pickering@gmail.com <mailto:jason.p.pickering@gmail.com mailto:jason.p.pickering@gmail.com > >
Date: Tue, 24 Sep 2013 10:26:56

To: <dhis2-devs@lists.launchpad.net mailto:dhis2-devs@lists.launchpad.net <mailto:dhis2-devs@lists.launchpad.net mailto:dhis2-devs@lists.launchpad.net > >
Subject: [Dhis2-devs] Upgrade from 2.10 to 2.12 failing

Using the latest 2.12 build from Jenkins.

Looks like something is failing on the startup, but not sure what?

This DB runs fine on 2.10. The first time around, a bunch of timestamps were updated. I killed the server, restarted, but still getting stuck at step 4 /10.

Any tips?

Regards,

Jason

  • INFO 2013-09-24 11:21:23,080 Found the following message senders: [org.hisp.dhis.message.EmailMessageSender@17a220d7, org.hisp.dhis.sms.outbound.SmsSender@151a0f60] (DefaultMessageService.java [main])

  • INFO 2013-09-24 11:21:39,093 Camel context started (DHIS2LifecycleStrategy.java [main])

  • INFO 2013-09-24 11:21:39,094 Searching for routes in directory: /home/testinstance/dhis_home/routes (DHIS2LifecycleStrategy.java [main])

  • INFO 2013-09-24 11:21:40,242 Executing startup routine [1 of 10, runlevel 1]: TableAlteror (DefaultStartupRoutineExecutor.java [main])

  • INFO 2013-09-24 11:21:46,749 Tables updated (TableAlteror.java [main])

  • INFO 2013-09-24 11:21:46,750 Executing startup routine [2 of 10, runlevel 1]: MapViewUpgrader (DefaultStartupRoutineExecutor.java [main])

  • INFO 2013-09-24 11:21:47,012 Executing startup routine [3 of 10, runlevel 2]: IdentityPopulator (DefaultStartupRoutineExecutor.java [main])

Sep 24, 2013 11:21:48 AM org.apache.catalina.core.StandardContext start

SEVERE: Error listenerStart

Sep 24, 2013 11:21:48 AM org.apache.catalina.core.StandardContext start

SEVERE: Context [/testinstance] startup failed due to previous errors

  • INFO 2013-09-24 11:21:48,017 De-registering jdbc driver: org.h2.Driver@65b8b5cd (StartupListener.java [main])

  • INFO 2013-09-24 11:21:48,018 De-registering jdbc driver: org.hsqldb.jdbc.JDBCDriver@2830803a (StartupListener.java [main])

  • INFO 2013-09-24 11:21:48,018 De-registering jdbc driver: com.mysql.jdbc.Driver@1ccdf3c2 (StartupListener.java [main])

  • INFO 2013-09-24 11:21:48,018 De-registering jdbc driver: org.postgresql.Driver@2f1261b1 (StartupListener.java [main])

  • INFO 2013-09-24 11:21:48,186 Camel context stopped (DHIS2LifecycleStrategy.java [main])

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads

SEVERE: A web application appears to have started a thread named [Resource Destroyer in BasicResourcePool.close()] but has failed to stop it. This is very likely to create a memory leak.

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Transactional resources]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Transaction synchronizations]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Prototype beans currently in creation]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

SEVERE: A web application created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@9db900f]) and a value of type [org.springframework.security.core.context.SecurityContextImpl] (value [org.springframework.security.core.context.SecurityContextImpl@ffffffff: Null authentication]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Current transaction name]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Current aspect-driven transaction]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Prototype beans currently in creation]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Locale context]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Current transaction read-only status]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Current transaction isolation level]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Actual transaction active]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

SEVERE: A web application created a ThreadLocal with key of type [null] (value [com.sun.xml.bind.v2.ClassFactory$1@642bf5bf]) and a value of type [java.util.WeakHashMap] (value [{class org.apache.camel.spring.CamelProducerTemplateFactoryBean=java.lang.ref.WeakReference@32b7e05f, class org.apache.camel.model.PackageScanDefinition=java.lang.ref.WeakReference@45eb96fc, class javax.xml.bind.annotation.adapters.CollapsedStringAdapter=java.lang.ref.WeakReference@6e059e40, class java.util.ArrayList=java.lang.ref.WeakReference@32d8b42e, class org.apache.camel.spring.CamelContextFactoryBean=java.lang.ref.WeakReference@311d5216}]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedInheritableThreadLocal] (value [Locale context]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Sep 24, 2013 11:21:48 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedInheritableThreadLocal] (value [Locale context]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Sep 24, 2013 11:21:49 AM org.apache.coyote.http11.Http11AprProtocol start

INFO: Starting Coyote HTTP/1.1 on http-8082

Sep 24, 2013 11:21:49 AM org.apache.catalina.startup.Catalina start

INFO: Server startup in 77181 ms