We are currently experiencing an issue when making use of the API for capturing Aggregate data.
Details of DHIS2 instance.
Docker image from dhis2/core: version 2.34
https://hub.docker.com/layers/dhis2/core/2.34.0/images/sha256-91816755a9fd476d874a5fbdd6c409c5441751f79140243b94eacd727f49ca17?context=explore
We are starting from a clean DHIS2 instance and importing our metadata config for setting up the Aggregate dataset etc.
We are able to manually add aggregate data onto the dataset without any issues.
The problem we are encountering is adding Aggregate data via the API which doesnt yet have a period created for the incoming period value.
When trying to capture the aggregate data via the API, on the first request it fails as seen in the below logs. It fails because the relevant period doesnt yet exist (Even though it states its being created). On the second try of the same request it goes through successfully. Any follow up request on the same period then also succeeds.
When a new request is sent through with a different period (one that doesnt yet exist), the same error is being thrown as the period doesnt yet exist when trying to insert the data.
Not entirely sure if this a bug with the version we are using, or just something we are forgetting to do as part of the initialization of the instance
Logs
- INFO 2020-06-22T13:56:39,581 Login attempt: admin (TwoFactorAuthenticationProvider.java [http-nio-8080-exec-4])
- WARN 2020-06-22T13:56:39,708 Authentication event AuthenticationSuccessEvent: admin; ip: 172.23.0.11 (AuthenticationLoggerListener.java [http-nio-8080-exec-4])
- INFO 2020-06-22T13:56:39,715 Starting data value import, options: ImportOptions{idSchemes=IdSchemes{idScheme=IdScheme{identifiableProperty=UID, attribute=null}, dataElementIdScheme=null, categoryOptionComboIdScheme=null, categoryOptionIdScheme=null, orgUnitIdScheme=null, programIdScheme=null, programStageIdScheme=null, trackedEntityIdScheme=null, trackedEntityAttributeIdScheme=null, dataSetIdScheme=null, attributeOptionComboIdScheme=null, programStageInstanceIdScheme=null}, dryRun=false, preheatCache=null, async=false, importStrategy=CREATE_AND_UPDATE, mergeMode=REPLACE, skipExistingCheck=false, ignoreEmptyCollection=false, sharing=false, skipNotifications=false, datasetAllowsPeriods=false, strictPeriods=false, strictDataElements=false, strictCategoryOptionCombos=false, strictAttributeOptionCombos=false, strictOrganisationUnits=false, requireCategoryOptionCombo=false, requireAttributeOptionCombo=false, force=false, firstRowIsHeader=true, skipLastUpdated=false}: 00:00:00.000 (Clock.java [http-nio-8080-exec-4])
- INFO 2020-06-22T13:56:39,716 Is ISO calendar: true, skip lock exception check: true (DefaultDataValueSetService.java [http-nio-8080-exec-4])
- INFO 2020-06-22T13:56:39,717 Skip audit: false, has authority to skip: true (DefaultDataValueSetService.java [http-nio-8080-exec-4])
- INFO 2020-06-22T13:56:39,718 Import options: ImportOptions{idSchemes=IdSchemes{idScheme=IdScheme{identifiableProperty=UID, attribute=null}, dataElementIdScheme=null, categoryOptionComboIdScheme=null, categoryOptionIdScheme=null, orgUnitIdScheme=null, programIdScheme=null, programStageIdScheme=null, trackedEntityIdScheme=null, trackedEntityAttributeIdScheme=null, dataSetIdScheme=null, attributeOptionComboIdScheme=null, programStageInstanceIdScheme=null}, dryRun=false, preheatCache=null, async=false, importStrategy=CREATE_AND_UPDATE, mergeMode=REPLACE, skipExistingCheck=false, ignoreEmptyCollection=false, sharing=false, skipNotifications=false, datasetAllowsPeriods=false, strictPeriods=false, strictDataElements=false, strictCategoryOptionCombos=false, strictAttributeOptionCombos=false, strictOrganisationUnits=false, requireCategoryOptionCombo=false, requireAttributeOptionCombo=false, force=false, firstRowIsHeader=true, skipLastUpdated=false} (DefaultDataValueSetService.java [http-nio-8080-exec-4])
- INFO 2020-06-22T13:56:39,718 Data value set identifier scheme: IdScheme{identifiableProperty=null, attribute=null}, data element: IdScheme{identifiableProperty=null, attribute=null}, org unit: IdScheme{identifiableProperty=null, attribute=null}, category option combo: IdScheme{identifiableProperty=null, attribute=null}, data set: IdScheme{identifiableProperty=null, attribute=null} (DefaultDataValueSetService.java [http-nio-8080-exec-4])
- INFO 2020-06-22T13:56:39,718 Identifier scheme: IdScheme{identifiableProperty=UID, attribute=null}, data element: IdScheme{identifiableProperty=UID, attribute=null}, org unit: IdScheme{identifiableProperty=UID, attribute=null}, category option combo: IdScheme{identifiableProperty=UID, attribute=null}, data set: IdScheme{identifiableProperty=UID, attribute=null} (DefaultDataValueSetService.java [http-nio-8080-exec-4])
- INFO 2020-06-22T13:56:39,722 ‘system-process’ create org.hisp.dhis.period.Period, name: 202003, uid: 202003 (AuditLogUtil.java [http-nio-8080-exec-4])
- INFO 2020-06-22T13:56:39,733 Validated outer meta-data: 00:00:00.018 (Clock.java [http-nio-8080-exec-4])
- ERROR 2020-06-22T13:56:39,807 java.lang.RuntimeException: Failed to flush BatchHandler
at org.hisp.quick.batchhandler.AbstractBatchHandler.flush(AbstractBatchHandler.java:321)
at org.hisp.dhis.dxf2.datavalueset.DefaultDataValueSetService.saveDataValueSet(DefaultDataValueSetService.java:1399)
at org.hisp.dhis.dxf2.datavalueset.DefaultDataValueSetService.saveDataValueSetJson(DefaultDataValueSetService.java:645)
at org.hisp.dhis.dxf2.datavalueset.DefaultDataValueSetService.saveDataValueSetJson(DefaultDataValueSetService.java:609)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:366)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:99)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
at com.sun.proxy.$Proxy247.saveDataValueSetJson(Unknown Source)
at org.hisp.dhis.webapi.controller.DataValueSetController.postJsonDataValueSet(DataValueSetController.java:291)
at org.hisp.dhis.webapi.controller.DataValueSetController$$FastClassBySpringCGLIB$$76a7ebbb.invoke()
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:769)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:747)
at org.springframework.security.access.intercept.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:69)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:747)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:689)
at org.hisp.dhis.webapi.controller.DataValueSetController$$EnhancerBySpringCGLIB$$681c346b.postJsonDataValueSet()
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:106)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:888)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:793)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:660)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.hisp.dhis.webapi.config.WebMvcMetricsConfig$PassThroughWebMvcMetricsFilter.doFilterInternal(WebMvcMetricsConfig.java:99)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.hisp.dhis.servlet.filter.RequestIdentifierFilter.doFilterInternal(RequestIdentifierFilter.java:103)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:320)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:126)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:90)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:118)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:158)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:203)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.hisp.dhis.security.filter.CorsFilter.doFilter(CorsFilter.java:91)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.hisp.dhis.security.filter.CustomAuthenticationFilter.doFilter(CustomAuthenticationFilter.java:69)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.oauth2.provider.authentication.OAuth2AuthenticationProcessingFilter.doFilter(OAuth2AuthenticationProcessingFilter.java:180)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.hisp.dhis.security.filter.AutomaticAccessFilter.doFilter(AutomaticAccessFilter.java:115)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:92)
at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:77)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.session.ConcurrentSessionFilter.doFilter(ConcurrentSessionFilter.java:152)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:215)
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:178)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.hisp.dhis.servlet.filter.HttpUrlPatternFilter.doFilter(HttpUrlPatternFilter.java:120)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.hisp.dhis.servlet.filter.ExcludableShallowEtagHeaderFilter.doFilterInternal(ExcludableShallowEtagHeaderFilter.java:108)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.orm.hibernate5.support.OpenSessionInViewFilter.doFilterInternal(OpenSessionInViewFilter.java:156)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:543)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:678)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:609)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:810)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1623)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.postgresql.util.PSQLException: ERROR: insert or update on table “datavalue” violates foreign key constraint “fk_datavalue_periodid”
Detail: Key (periodid)=(15537) is not present in table “period”.
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2440)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2183)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:308)
at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365)
at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:307)
at org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:293)
at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:270)
at org.postgresql.jdbc.PgStatement.executeUpdate(PgStatement.java:244)
at org.hisp.quick.batchhandler.AbstractBatchHandler.flush(AbstractBatchHandler.java:310)
… 146 more
(DefaultDataValueSetService.java [http-nio-8080-exec-4])