after upgrading from 2.35.2 to 2.40. I ran the analytics tables from data administration but unfortunately, I have found many of errors as the following:
1- ERROR 2023-06-25T22:02:32,680 [ANALYTICS_TABLE gL58x8tssZX] Item failed after 0.004s: Cannot invoke “Object.equals(Object)” because the return value of “java.util.Map.get(Object)” is null (ControlledJobProgress.java [ForkJoinPool-43-worker-5])
java.lang.NullPointerException: Cannot invoke “Object.equals(Object)” because the return value of “java.util.Map.get(Object)” is null.
2- RROR 2023-06-25T22:02:32,713 processing aborted: skipped stage, failing item caused abort. parallel processing aborted after 11 successful and 1 failed items (NotificationLoggerUtil.java [taskScheduler-13]) UID:gL58x8tssZX
ERROR 2023-06-25T22:02:32,718 [ANALYTICS_TABLE gL58x8tssZX] Stage failed after 0.045s: processing aborted: skipped stage, failing item caused abort. parallel processing aborted after 11 successful and 1 failed items (ControlledJobProgress.java [taskScheduler-13]) UID:gL58x8tssZX
java.lang.RuntimeException: processing aborted: skipped stage, failing item caused abort. parallel processing aborted after 11 successful and 1 failed items
3- ERROR 2023-06-25T22:02:32,719 Process failed: java.util.concurrent.CancellationException (NotificationLoggerUtil.java [taskScheduler-13]) UID:gL58x8tssZX
when I run event visulizer I got the below error from catalina as the following:
INFO 2023-06-25T22:25:42,716 Query failed, likely because the requested analytics table does not exist (AbstractJdbcEventAnalyticsManager.java [http-nio-8081-exec-6]) IDY7v4M13PcRFk+/Ro0RG6MNqv5kj7Adhl7JuShP7NT0Q=
org.springframework.jdbc.BadSqlGrammarException: StatementCallback; bad SQL grammar [select count(ax.“psi”) as value,ax.“monthly”,ax.“SpJCIRJo7Fp” from analytics_event_n5dqvirkajs as ax where (ax.“monthly” in (‘202206’, ‘202207’, ‘202208’, ‘202209’, ‘202210’, ‘202211’, ‘202212’, ‘202301’, ‘202302’, ‘202303’, ‘202304’, ‘202305’) )and ax.“uidlevel1” in (‘AnO97v1hLDU’) and ax.“ps” = ‘bowLYLheCxP’ and ax.“yearly” in (‘2022’, ‘2023’) group by ax.“monthly”, ax.“SpJCIRJo7Fp” order by value desc ]; nested exception is org.postgresql.util.PSQLException: ERROR: relation “analytics_event_n5dqvirkajs” does not exist
Position: 68
caused by: org.postgresql.util.PSQLException: ERROR: relation “analytics_event_n5dqvirkajs” does not exist
Position: 68
INFO 2023-06-25T22:25:42,717 Query failed, likely because the requested analytics table does not exist (AbstractJdbcEventAnalyticsManager.java [http-nio-8081-exec-6]) IDY7v4M13PcRFk+/Ro0RG6MNqv5kj7Adhl7JuShP7NT0Q=
org.springframework.jdbc.BadSqlGrammarException: StatementCallback; bad SQL grammar [select count(ax.“psi”) as value,ax.“yearly”,ax.“SpJCIRJo7Fp” from analytics_event_n5dqvirkajs as ax where (ax.“yearly” in (‘2023’) )and ax.“uidlevel1” in (‘AnO97v1hLDU’) and ax.“ps” = ‘bowLYLheCxP’ and ax.“yearly” in (‘2023’) group by ax.“yearly”, ax.“SpJCIRJo7Fp” order by value desc ]; nested exception is org.postgresql.util.PSQLException: ERROR: relation “analytics_event_n5dqvirkajs” does not exist
Position: 67
Caused by: org.postgresql.util.PSQLException: ERROR: relation “analytics_event_n5dqvirkajs” does not exist
Position: 67
many of error showed in catalina when I run event report as the following:
INFO 2023-06-25T22:32:48,351 Query failed, likely because the requested analytics table does not exist (AbstractJdbcEventAnalyticsManager.java [http-nio-8081-exec-8]) IDY7v4M13PcRFk+/Ro0RG6MNqv5kj7Adhl7JuShP7NT0Q=
org.springframework.jdbc.BadSqlGrammarException: StatementCallback; bad SQL grammar [select count(distinct tei) as value from analytics_enrollment_n5dqvirkajs as ax where (enrollmentdate < cast( ‘2024-01-01’ as date ) and enrollmentdate >= cast( ‘2023-01-01’ as date ))and (ax.“uidlevel1” = ‘AnO97v1hLDU’ ) and (coalesce(“SpJCIRJo7Fp”::text,‘’) = ‘مركزي’) ]; nested exception is org.postgresql.util.PSQLException: ERROR: relation “analytics_enrollment_n5dqvirkajs” does not exist
Position: 42
Caused by: org.postgresql.util.PSQLException: ERROR: relation “analytics_enrollment_n5dqvirkajs” does not exist
Position: 42
org.springframework.jdbc.UncategorizedSQLException: StatementCallback; uncategorized SQLException for SQL [select count(distinct tei) as value from analytics_enrollment_n5dqvirkajs as ax where (enrollmentdate < cast( ‘2024-01-01’ as date ) and enrollmentdate >= cast( ‘2023-01-01’ as date ))and (ax.“uidlevel1” = ‘AnO97v1hLDU’ ) and (coalesce(“SpJCIRJo7Fp”::text,‘’) = ‘مديرية’) ]; SQL state [25P02]; error code [0]; ERROR: current transaction is aborted, commands ignored until end of transaction block; nested exception is org.postgresql.util.PSQLException: ERROR: current transaction is aborted, commands ignored until end of transaction block
Caused by: org.postgresql.util.PSQLException: ERROR: current transaction is aborted, commands ignored until end of transaction block
Caused by: org.postgresql.util.PSQLException: ERROR: relation “analytics_enrollment_n5dqvirkajs” does not exist
Position: 42
rg.springframework.jdbc.BadSqlGrammarException: StatementCallback; bad SQL grammar [select count(psi) as count, ST_Extent(coalesce(ax.“FCyRjQpoHMd”)) as extent from analytics_event_n5dqvirkajs as ax where ((ax.“executiondate” >= ‘2023-01-01’ and ax.“executiondate” < ‘2024-01-01’))and ax.“uidlevel1” in (‘AnO97v1hLDU’) and ax.“ps” = ‘CokmvJY94M9’ and coalesce(ax.“FCyRjQpoHMd”) is not null and ax.“yearly” in (‘2023’) ]; nested exception is org.postgresql.util.PSQLException: ERROR: relation “analytics_event_n5dqvirkajs” does not exist
Position: 83
Caused by: org.postgresql.util.PSQLException: ERROR: relation “analytics_event_n5dqvirkajs” does not exist
Position: 83
org.springframework.jdbc.BadSqlGrammarException: StatementCallback; bad SQL grammar [select count(psi) as count, ST_Extent(coalesce(ax.“FCyRjQpoHMd”)) as extent from analytics_event_n5dqvirkajs as ax where ((ax.“executiondate” >= ‘2023-01-01’ and ax.“executiondate” < ‘2024-01-01’))and ax.“uidlevel1” in (‘AnO97v1hLDU’) and ax.“ps” = ‘CokmvJY94M9’ and coalesce(ax.“FCyRjQpoHMd”) is not null and ax.“yearly” in (‘2023’) ]; nested exception is org.postgresql.util.PSQLException: ERROR: relation “analytics_event_n5dqvirkajs” does not exist
Position: 83
Caused by: org.postgresql.util.PSQLException: ERROR: relation “analytics_event_n5dqvirkajs” does not exist
Position: 83
INFO 2023-06-25T22:33:01,302 Query failed, likely because the requested analytics table does not exist (AbstractJdbcEventAnalyticsManager.java [http-nio-8081-exec-2]) IDY7v4M13PcRFk+/Ro0RG6MNqv5kj7Adhl7JuShP7NT0Q=
org.springframework.jdbc.BadSqlGrammarException: StatementCallback; bad SQL grammar [select psi,ps,executiondate,storedby,createdbydisplayname,lastupdatedbydisplayname,lastupdated,duedate,enrollmentdate,incidentdate,tei,pi,ST_AsGeoJSON(psigeometry, 6) as geometry,longitude,latitude,ouname,ounamehierarchy,oucode,pistatus,psistatus,ax.“ou”,ax.“SpJCIRJo7Fp”,ax.“ANwhVTDHT3b”,ax.“JDTKUL5zk1o”,ax.“ICCSfrdg017” as “bowLYLheCxP.ICCSfrdg017”,ax.“ICCSfrdg007” as “bowLYLheCxP.ICCSfrdg007”,ax.“ICCSfrdg010” as “bowLYLheCxP.ICCSfrdg010”,ax.“cGJPquZg6ZD” as “bowLYLheCxP.cGJPquZg6ZD”,ax.“ICCSfrdg011” as “bowLYLheCxP.ICCSfrdg011”,ax.“ICCSfrdg023” as “bowLYLheCxP.ICCSfrdg023”,ax.“ICCSfrdg002” as “bowLYLheCxP.ICCSfrdg002”,ax.“XVUR0f0JdD0” as “bowLYLheCxP.XVUR0f0JdD0”,ax.“ICCSfrdg012” as “bowLYLheCxP.ICCSfrdg012”,ax.“ICCSfrdg004” as “bowLYLheCxP.ICCSfrdg004” from analytics_event_n5dqvirkajs as ax where ((ax.“executiondate” >= ‘2023-01-01’ and ax.“executiondate” < ‘2024-01-01’))and ax.“uidlevel1” in (‘AnO97v1hLDU’) and ax.“ps” = ‘bowLYLheCxP’ and ax.“ICCSfrdg012” in (‘بحاجة الى اصلاح’) and ax.“yearly” in (‘2023’) order by “executiondate” desc nulls last limit 101 offset 0]; nested exception is org.postgresql.util.PSQLException: ERROR: relation “analytics_event_n5dqvirkajs” does not exist
Position: 773
caused by: org.postgresql.util.PSQLException: ERROR: relation “analytics_event_n5dqvirkajs” does not exist
Position: 773
These issues could be caused by many things so it’s not easy to asses without testing and checking the complete log (without sensitive info); however, it might help if you completely clear the analytics tables first from Data Administration → Maintenance and then after that run the analytics tables exports (make sure you have enough disk space).
If the above doesn’t work, please share the complete Catalina.out log (without sensitive info).
Finally, to double check that when you performed the upgrade, you’d followed the instructions such as “[dropping] any materialized SQL views from your database” (Upgrade)
part#1 of cataline log:
I have completely cleared the analytics tables and exported them again and then I monitored cataline loge and many of errors appered as the following:
INFO 2023-06-27T01:06:41,869 Last successful analytics table update: ‘2023-05-28T03:00:00’ (DefaultAnalyticsTableGenerator.java [taskScheduler-7]) UID:Cdakta7Hm6O
INFO 2023-06-27T01:06:41,870 Analytics table update process (NotificationLoggerUtil.java [taskScheduler-7]) UID:Cdakta7Hm6O
INFO 2023-06-27T01:06:41,870 [ANALYTICS_TABLE Cdakta7Hm6O] Process started: Analytics table update process (ControlledJobProgress.java [taskScheduler-7]) UID:Cdakta7Hm6O
INFO 2023-06-27T01:06:41,872 Drop SQL views (NotificationLoggerUtil.java [taskScheduler-7]) UID:Cdakta7Hm6O
INFO 2023-06-27T01:06:41,872 [ANALYTICS_TABLE Cdakta7Hm6O] Stage started: Drop SQL views (ControlledJobProgress.java [taskScheduler-7]) UID:Cdakta7Hm6O
INFO 2023-06-27T01:06:41,892 13 successful and 0 failed items (NotificationLoggerUtil.java [taskScheduler-7]) UID:Cdakta7Hm6O
INFO 2023-06-27T01:06:41,893 [ANALYTICS_TABLE Cdakta7Hm6O] Stage completed after 0.021s: 13 successful and 0 failed items (ControlledJobProgress.java [taskScheduler-7]) UID:Cdakta7Hm6O
INFO 2023-06-27T01:06:41,893 Generating resource tables (NotificationLoggerUtil.java [taskScheduler-7]) UID:Cdakta7Hm6O
WARN 2023-06-27T01:06:50,588 Validation failed for SQL view ‘HnZAdad7N37’ with code: ‘E4301’ and message: ‘SQL query must be a select query’ (DefaultSqlViewService.java [taskScheduler-7]) UID:Cdakta7Hm6O
WARN 2023-06-27T01:06:50,589 Ignoring SQL view which failed validation: HnZAdad7N37, users, message: SQL query must be a select query (DefaultResourceTableService.java [taskScheduler-7]) UID:Cdakta7Hm6O
INFO 2023-06-27T01:06:50,593 13 successful and 0 failed items (NotificationLoggerUtil.java [taskScheduler-7]) UID:Cdakta7Hm6O
INFO 2023-06-27T01:06:50,593 [ANALYTICS_TABLE Cdakta7Hm6O] Stage completed after 0.025s: 13 successful and 0 failed items (ControlledJobProgress.java [taskScheduler-7]) UID:Cdakta7Hm6O
WARN 2023-06-27T01:06:58,147 Validation failed for SQL view ‘HnZAdad7N37’ with code: ‘E4301’ and message: ‘SQL query must be a select query’ (DefaultSqlViewService.java [taskScheduler-7]) UID:Cdakta7Hm6O
WARN 2023-06-27T01:06:58,147 Ignoring SQL view which failed validation: HnZAdad7N37, users, message: SQL query must be a select query (DefaultResourceTableService.java [taskScheduler-7]) UID:Cdakta7Hm6O
INFO 2023-06-27T01:06:58,150 13 successful and 0 failed items (NotificationLoggerUtil.java [taskScheduler-7]) UID:Cdakta7Hm6O
INFO 2023-06-27T01:06:58,150 [ANALYTICS_TABLE Cdakta7Hm6O] Stage completed after 0.021s: 13 successful and 0 failed items (ControlledJobProgress.java [taskScheduler-7]) UID:Cdakta7Hm6O
INFO 2023-06-27T01:06:58,150 Table update done: analytics: 00:00:07.555 (Clock.java [taskScheduler-7]) UID:Cdakta7Hm6O
WARN 2023-06-27T01:06:58,637 Validation failed for SQL view ‘HnZAdad7N37’ with code: ‘E4301’ and message: ‘SQL query must be a select query’ (DefaultSqlViewService.java [taskScheduler-7]) UID:Cdakta7Hm6O
WARN 2023-06-27T01:06:58,637 Ignoring SQL view which failed validation: HnZAdad7N37, users, message: SQL query must be a select query (DefaultResourceTableService.java [taskScheduler-7]) UID:Cdakta7Hm6O
INFO 2023-06-27T01:06:58,641 13 successful and 0 failed items (NotificationLoggerUtil.java [taskScheduler-7]) UID:Cdakta7Hm6O
INFO 2023-06-27T01:06:58,641 [ANALYTICS_TABLE Cdakta7Hm6O] Stage completed after 0.018s: 13 successful and 0 failed items (ControlledJobProgress.java [taskScheduler-7]) UID:Cdakta7Hm6O
INFO 2023-06-27T01:06:58,641 Table update done: analytics_completeness: 00:00:00.490 (Clock.java [taskScheduler-7]) UID:Cdakta7Hm6O
INFO 2023-06-27T01:06:58,645 Populate table task number: 1 (DefaultAnalyticsTableService.java [taskScheduler-7]) UID:Cdakta7Hm6O
INFO 2023-06-27T01:06:58,818 Populate analytics_completenesstarget_temp in: 0.172825 sec. (AbstractJdbcTableManager.java [taskScheduler-7]) UID:Cdakta7Hm6O
INFO 2023-06-27T01:06:58,818 1 successful and 0 failed items (NotificationLoggerUtil.java [taskScheduler-7]) UID:Cdakta7Hm6O
INFO 2023-06-27T01:06:58,818 [ANALYTICS_TABLE Cdakta7Hm6O] Stage completed after 0.173s: 1 successful and 0 failed items (ControlledJobProgress.java [taskScheduler-7]) UID:Cdakta7Hm6O
INFO 2023-06-27T01:06:58,819 Populated analytics tables: 00:00:00.177 (Clock.java [taskScheduler-7]) UID:Cdakta7Hm6O
INFO 2023-06-27T01:06:58,819 Invoking analytics table hooks COMPLETENESS_TARGET (NotificationLoggerUtil.java [taskScheduler-7]) UID:Cdakta7Hm6O
WARN 2023-06-27T01:06:59,341 Validation failed for SQL view ‘HnZAdad7N37’ with code: ‘E4301’ and message: ‘SQL query must be a select query’ (DefaultSqlViewService.java [taskScheduler-7]) UID:Cdakta7Hm6O
WARN 2023-06-27T01:06:59,342 Ignoring SQL view which failed validation: HnZAdad7N37, users, message: SQL query must be a select query (DefaultResourceTableService.java [taskScheduler-7]) UID:Cdakta7Hm6O
INFO 2023-06-27T01:06:59,345 13 successful and 0 failed items (NotificationLoggerUtil.java [taskScheduler-7]) UID:Cdakta7Hm6O
INFO 2023-06-27T01:06:59,345 [ANALYTICS_TABLE Cdakta7Hm6O] Stage completed after 0.021s: 13 successful and 0 failed items (ControlledJobProgress.java [taskScheduler-7]) UID:Cdakta7Hm6O
INFO 2023-06-27T01:06:59,345 Table update done: analytics_completenesstarget: 00:00:00.703 (Clock.java [taskScheduler-7]) UID:Cdakta7Hm6O
WARN 2023-06-27T01:06:59,517 Validation failed for SQL view ‘HnZAdad7N37’ with code: ‘E4301’ and message: ‘SQL query must be a select query’ (DefaultSqlViewService.java [taskScheduler-7]) UID:Cdakta7Hm6O
WARN 2023-06-27T01:06:59,518 Ignoring SQL view which failed validation: HnZAdad7N37, users, message: SQL query must be a select query (DefaultResourceTableService.java [taskScheduler-7]) UID:Cdakta7Hm6O
INFO 2023-06-27T01:06:59,521 13 successful and 0 failed items (NotificationLoggerUtil.java [taskScheduler-7]) UID:Cdakta7Hm6O
INFO 2023-06-27T01:06:59,521 [ANALYTICS_TABLE Cdakta7Hm6O] Stage completed after 0.018s: 13 successful and 0 failed items (ControlledJobProgress.java [taskScheduler-7]) UID:Cdakta7Hm6O
INFO 2023-06-27T01:06:59,521 Table update done: analytics_orgunittarget: 00:00:00.175 (Clock.java [taskScheduler-7]) UID:Cdakta7Hm6O
INFO 2023-06-27T01:06:59,573 Populate table task number: 7 (DefaultAnalyticsTableService.java [taskScheduler-7]) UID:Cdakta7Hm6O
INFO 2023-06-27T01:06:59,579 OwnershipAnalytics query row count was 0 for analytics_ownership_temp_xmu2sb8oetv (JdbcOwnershipAnalyticsTableManager.java [ForkJoinPool-30-worker-3])
INFO 2023-06-27T01:06:59,579 OwnershipAnalytics query row count was 0 for analytics_ownership_temp_xzvwv7zjzm2 (JdbcOwnershipAnalyticsTableManager.java [ForkJoinPool-30-worker-1])
INFO 2023-06-27T01:06:59,580 Populate analytics_ownership_temp_xmu2sb8oetv in: 0.004577 sec. (JdbcOwnershipAnalyticsTableManager.java [ForkJoinPool-30-worker-3])
INFO 2023-06-27T01:06:59,580 Populate analytics_ownership_temp_xzvwv7zjzm2 in: 0.004597 sec. (JdbcOwnershipAnalyticsTableManager.java [ForkJoinPool-30-worker-1])
ERROR 2023-06-27T01:06:59,580 Cannot invoke “Object.equals(Object)” because the return value of “java.util.Map.get(Object)” is null (NotificationLoggerUtil.java [ForkJoinPool-30-worker-5])
WARN 2023-06-27T01:06:59,580 [ANALYTICS_TABLE Cdakta7Hm6O] Process aborted after 17.71s: aborted after error: Cannot invoke “Object.equals(Object)” because the return value of “java.util.Map.get(Object)” is null (ControlledJobProgress.java [ForkJoinPool-30-worker-5])
ERROR 2023-06-27T01:06:59,589 [ANALYTICS_TABLE Cdakta7Hm6O] Item failed after 0.006s: Cannot invoke “Object.equals(Object)” because the return value of “java.util.Map.get(Object)” is null (ControlledJobProgress.java [ForkJoinPool-30-worker-5])
java.lang.NullPointerException: Cannot invoke “Object.equals(Object)” because the return value of “java.util.Map.get(Object)” is null
at org.hisp.dhis.analytics.table.JdbcOwnershipWriter.sameValue(JdbcOwnershipWriter.java:209) ~[dhis-service-analytics-2.40.0.jar:?]
at org.hisp.dhis.analytics.table.JdbcOwnershipWriter.write(JdbcOwnershipWriter.java:110) ~[dhis-service-analytics-2.40.0.jar:?]
at org.hisp.dhis.analytics.table.JdbcOwnershipAnalyticsTableManager.lambda$populateTableInternal$1(JdbcOwnershipAnalyticsTableManager.java:182) ~[dhis-service-analytics-2.40.0.jar:?]
at org.springframework.jdbc.core.JdbcTemplate$RowCallbackHandlerResultSetExtractor.extractData(JdbcTemplate.java:1696) ~[spring-jdbc-5.3.25.jar:5.3.25]
at org.springframework.jdbc.core.JdbcTemplate$1QueryStatementCallback.doInStatement(JdbcTemplate.java:453) ~[spring-jdbc-5.3.25.jar:5.3.25]
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:381) ~[spring-jdbc-5.3.25.jar:5.3.25]
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:465) ~[spring-jdbc-5.3.25.jar:5.3.25]
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:470) ~[spring-jdbc-5.3.25.jar:5.3.25]
at org.hisp.dhis.analytics.table.JdbcOwnershipAnalyticsTableManager.populateTableInternal(JdbcOwnershipAnalyticsTableManager.java:181) ~[dhis-service-analytics-2.40.0.jar:?]
at org.hisp.dhis.analytics.table.JdbcOwnershipAnalyticsTableManager.populateTable(JdbcOwnershipAnalyticsTableManager.java:160) ~[dhis-service-analytics-2.40.0.jar:?]
at org.hisp.dhis.analytics.table.AbstractJdbcTableManager.populateTablePartition(AbstractJdbcTableManager.java:255) ~[dhis-service-analytics-2.40.0.jar:?]
at jdk.internal.reflect.GeneratedMethodAccessor1458.invoke(Unknown Source) ~[?:?]
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
at java.lang.reflect.Method.invoke(Method.java:568) ~[?:?]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) ~[spring-aop-5.3.25.jar:5.3.25]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208) ~[spring-aop-5.3.25.jar:5.3.25]
at jdk.proxy3.$Proxy372.populateTablePartition(Unknown Source) ~[?:?]
at org.hisp.dhis.analytics.table.DefaultAnalyticsTableService.lambda$populateTables$2(DefaultAnalyticsTableService.java:226) ~[dhis-service-analytics-2.40.0.jar:?]
at org.hisp.dhis.scheduling.JobProgress.lambda$runStageInParallel$5(JobProgress.java:498) ~[dhis-api-2.40.0.jar:?]
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) [?:?]
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625) [?:?]
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) [?:?]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) [?:?]
at java.util.stream.ReduceOps$ReduceTask.doLeaf(ReduceOps.java:960) [?:?]
at java.util.stream.ReduceOps$ReduceTask.doLeaf(ReduceOps.java:934) [?:?]
at java.util.stream.AbstractTask.compute(AbstractTask.java:327) [?:?]
at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:754) [?:?]
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) [?:?]
at java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) [?:?]
at java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) [?:?]
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) [?:?]
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) [?:?]
ERROR 2023-06-27T01:06:59,633 [ANALYTICS_TABLE Cdakta7Hm6O] Stage failed after 0.056s: processing aborted: skipped stage, failing item caused abort. parallel processing aborted after 11 successful and 1 failed items (ControlledJobProgress.java [taskScheduler-7]) UID:Cdakta7Hm6O
java.lang.RuntimeException: processing aborted: skipped stage, failing item caused abort. parallel processing aborted after 11 successful and 1 failed items
at org.hisp.dhis.scheduling.ControlledJobProgress.cancellationAsAbort(ControlledJobProgress.java:390) ~[dhis-service-core-2.40.0.jar:?]
at org.hisp.dhis.scheduling.ControlledJobProgress.failedStage(ControlledJobProgress.java:235) ~[dhis-service-core-2.40.0.jar:?]
at org.hisp.dhis.scheduling.JobProgress.autoSkipStage(JobProgress.java:382) ~[dhis-api-2.40.0.jar:?]
at org.hisp.dhis.scheduling.JobProgress.runStageInParallel(JobProgress.java:523) ~[dhis-api-2.40.0.jar:?]
at org.hisp.dhis.analytics.table.DefaultAnalyticsTableService.populateTables(DefaultAnalyticsTableService.java:225) ~[dhis-service-analytics-2.40.0.jar:?]
at org.hisp.dhis.analytics.table.DefaultAnalyticsTableService.update(DefaultAnalyticsTableService.java:137) ~[dhis-service-analytics-2.40.0.jar:?]
at org.hisp.dhis.analytics.table.DefaultAnalyticsTableGenerator.generateTables(DefaultAnalyticsTableGenerator.java:111) ~[dhis-service-analytics-2.40.0.jar:?]
at org.hisp.dhis.analytics.table.scheduling.AnalyticsTableJob.execute(AnalyticsTableJob.java:72) ~[dhis-service-analytics-2.40.0.jar:?]
at org.hisp.dhis.scheduling.AbstractSchedulingManager.execute(AbstractSchedulingManager.java:325) ~[dhis-service-core-2.40.0.jar:?]
at org.hisp.dhis.scheduling.DefaultSchedulingManager.lambda$executeNow$6(DefaultSchedulingManager.java:200) ~[dhis-service-core-2.40.0.jar:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) [?:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) [spring-context-5.3.25.jar:5.3.25]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) [?:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) [?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
at java.lang.Thread.run(Thread.java:833) [?:?]
INFO 2023-06-27T01:06:59,633 Populated analytics tables: 00:00:00.111 (Clock.java [taskScheduler-7]) UID:Cdakta7Hm6O
ERROR 2023-06-27T01:06:59,633 Process failed: java.util.concurrent.CancellationException (NotificationLoggerUtil.java [taskScheduler-7]) UID:Cdakta7Hm6O
ERROR 2023-06-27T01:06:59,633 Job failed: ‘inMemoryAnalyticsJob’ (AbstractSchedulingManager.java [taskScheduler-7]) UID:Cdakta7Hm6O
java.util.concurrent.CancellationException: null
at org.hisp.dhis.scheduling.ControlledJobProgress.startingStage(ControlledJobProgress.java:202) ~[dhis-service-core-2.40.0.jar:?]
at org.hisp.dhis.scheduling.JobProgress.startingStage(JobProgress.java:186) ~[dhis-api-2.40.0.jar:?]
at org.hisp.dhis.scheduling.JobProgress.startingStage(JobProgress.java:191) ~[dhis-api-2.40.0.jar:?]
at org.hisp.dhis.analytics.table.DefaultAnalyticsTableService.update(DefaultAnalyticsTableService.java:140) ~[dhis-service-analytics-2.40.0.jar:?]
at org.hisp.dhis.analytics.table.DefaultAnalyticsTableGenerator.generateTables(DefaultAnalyticsTableGenerator.java:111) ~[dhis-service-analytics-2.40.0.jar:?]
at org.hisp.dhis.analytics.table.scheduling.AnalyticsTableJob.execute(AnalyticsTableJob.java:72) ~[dhis-service-analytics-2.40.0.jar:?]
at org.hisp.dhis.scheduling.AbstractSchedulingManager.execute(AbstractSchedulingManager.java:325) ~[dhis-service-core-2.40.0.jar:?]
at org.hisp.dhis.scheduling.DefaultSchedulingManager.lambda$executeNow$6(DefaultSchedulingManager.java:200) ~[dhis-service-core-2.40.0.jar:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) [?:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) [spring-context-5.3.25.jar:5.3.25]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) [?:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) [?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
at java.lang.Thread.run(Thread.java:833) [?:?]
ERROR 2023-06-27T01:06:59,634 java.util.concurrent.CancellationException
at org.hisp.dhis.scheduling.ControlledJobProgress.startingStage(ControlledJobProgress.java:202)
at org.hisp.dhis.scheduling.JobProgress.startingStage(JobProgress.java:186)
at org.hisp.dhis.scheduling.JobProgress.startingStage(JobProgress.java:191)
at org.hisp.dhis.analytics.table.DefaultAnalyticsTableService.update(DefaultAnalyticsTableService.java:140)
at org.hisp.dhis.analytics.table.DefaultAnalyticsTableGenerator.generateTables(DefaultAnalyticsTableGenerator.java:111)
at org.hisp.dhis.analytics.table.scheduling.AnalyticsTableJob.execute(AnalyticsTableJob.java:72)
at org.hisp.dhis.scheduling.AbstractSchedulingManager.execute(AbstractSchedulingManager.java:325)
at org.hisp.dhis.scheduling.DefaultSchedulingManager.lambda$executeNow$6(DefaultSchedulingManager.java:200)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:833)
(AbstractSchedulingManager.java [taskScheduler-7]) UID:Cdakta7Hm6O
Thank you for the complete log. There was a bug which got fixed five days ago and it might be directly related to the issue you are facing: [DHIS2-15467] - Jira
However, if you don’t have null organizationunitid in that table then the cause of the error could still be for different things. First, most of the errors that were posted in the first log doesn’t seem to appear again which is good and indicates that you’d did a correct step such as droping the SQL views and clearing the analytics tables; however, you might still need to check again the following:
Please double check diskspace
Try to manually delete the tables (see post below)
Found a similar post (from 2018 but has relevant info) with general instructions from @Calle_Hedberg on which tables to delete: Analytics Failure in 2.33.1
You are correct - I have found a null value for the organizationunitid in that table. However, I am unsure about the next steps to address this issue. I have also reviewed the link you provided, but I am still unclear about how it can help me or what I should do next.
Can you please provide clear instructions on how to address this issue? I would greatly appreciate your guidance. Thank you.
I have reviewed the link you provided regarding this issue, and I came across one of the replies which recommended changing the JDK to version 20. After checking my current JDK version, I found out that it is version 17. To address this, I added the following line into the setenv.sh file:
However, I am unsure whether I should use “java-1.17.0-openjdk-amd64” or “java-17-openjdk-amd64”. Can you please clarify which one is correct? Thank you.
By the way, I tried two options, but none solved the issue.
@Gassim Could you help me solve this issue? Feel free to have access to my device via TeamViewer or any other app.
@Gassim good morning. Could you please help me overcome this issue? I appreciate your time and efforts.
Thanks! Like I mentioned, if there are null values in the table for the organizationunitid then this might be related to the bug issue. If it is then we will need to wait for the fix and probably you will need to update; however, if it’s not then it will still require more investigation to understand the issue.
Thank you for your patience. I will let you know if I find something out or if I have more questions.
Update:
@sami.oracle10g indeed, @maikel said the issue above is related to the bug which is now fixed later releases. Here are the tips from Maikel:
Regarding the last post above, yes, that seems really related to my last fix. It will avoid this problem in future releases
For now, the users can “Skip generation of organisation unit ownership data” in the Analytics Table Export
Assuming they don’t need any feature related to org units ownership
The export error “null” only happens in that step if I’m not wrong…
Or they can fix the null values in the table programownershiphistory
For any one of the solutions chosen, they have to run a full analytics export to make sure the state is consistent
@Gassim excuse me, there is no skip generation of organization unit ownership data in the Analytics Table export. see attached for further clarification.
