Bugs in DHIS new importer.

Hi

I will try and look into these issues, please be aware that I have done a lot of fixes to the importer in the last 2 days, so if this is from before that… probably you need to re-test.

Will go over them on Monday and see what can be fixed for release.

···

On Thu, Jun 23, 2016 at 7:28 PM, Nalinikanth Meesala nalinim@thoughtworks.com wrote:

Hi Lars & Morten,

I am working for MSF-OCA project from ThoughtWorks. As you are aware that we are building a metadata synchronisation. We had a bug bash recently and below are the bugs we found in new importer. We did the bug bash on trunk.

I am attaching a file with payloads and logs which might help you.

  • Sharing settings will sync only if you change any metadata of that particular element along with change in sharing settings.
  • When we create Custom attributes and make them mandatory and assign to a data set (for example) and sync them to local instance, we get a run time exception stating that attribute for dataset not given any value. This is because even if it is mandatory we are not changing the dataset to set the value to attribute.
  • Data set - when updated with a data approval workflow - does not sync to local. Everything syncs and association is not syncing
  • when we updated the existing program with adding combination of categories and synched.
  • while updating attributes in the program with registration.
  • ERROR 2016-06-21 21:23:24,325 ERROR: duplicate key value violates unique constraint ““programtrackedentityattribute_unique_key””

Detail: Key (programid, trackedentityattributeid)=(255, 240) already exists. (SqlExceptionHelper.java [taskScheduler-6])

  • ERROR 2016-06-21 21:23:24,385 Exception occurred while trying to import the metadata. could not execute statement (MetadataImportHandler.java [taskScheduler-6])

org.hibernate.exception.ConstraintViolationException: could not execute statement at org.hibernate.exception.internal.SQLStateConversionDelegate.convert(SQLStateConversionDelegate.java:112)"Error on Edit Program Stage page. Press cancel and exception is thrown. Analytics Export Error. Process failed. null.

  • Charts, pivot tables, Report Table/ Pivot Table, Event Report, Event Chart, Validation Rule

Metadata Sync failed for the above metadata With the below runtime exception.

— Exception occurred while trying to import the metadata. object references an unsaved transient instance - save the transient instance before flushing: org.hisp.dhis.dataelement.DataElement (MetadataSyncTask.java [taskScheduler-9])

  • Admin user when added to dashboard item is not syncing with metadat sync.
  • When deleted a legend at HQ and synced to local. The legend at Local also is deleted on UI (exists in DB). Also, legend is getting duplicated with a different UID in DB.
  • Metadata Filter (Import-Export --> Metadata Detailed Export)

modification to properties like name, description is successfully synced but the selection is not synced, the payload also have the details of the updated data when synced.

  • custom forms are synced properly when created, but while Updation of custom form not reflecting on UI. However, payload has the full information of updates.


Thanks & Regards,

Nalinikanth M

Quality Analyst

Email
nalinim@thoughtworks.com
Telephone
+91 9052234588
ThoughtWorks

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

Dear Morten,

Yes, I have the same issues but I wanted to clearly document them as a bug. This happens when you extract the program metadata and then try to import it to an existing similar program (same uid but in production).

The following fails to be imported correctly

  • Option values are not imported in process

  • message reminder templates are not imported

  • programTrackedEntityAttributes are not imported (seems they are imported before attributes or viceversa, need to confirm this)

  • data entry stage custom forms are not imported

Will document well and submit the bug (after testing against the new release).

Thanks.

Alex

···

On Fri, Jun 24, 2016 at 2:32 PM, Morten Olav Hansen morten@dhis2.org wrote:

Hi

I will try and look into these issues, please be aware that I have done a lot of fixes to the importer in the last 2 days, so if this is from before that… probably you need to re-test.

Will go over them on Monday and see what can be fixed for release.


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Thu, Jun 23, 2016 at 7:28 PM, Nalinikanth Meesala nalinim@thoughtworks.com wrote:

Hi Lars & Morten,

I am working for MSF-OCA project from ThoughtWorks. As you are aware that we are building a metadata synchronisation. We had a bug bash recently and below are the bugs we found in new importer. We did the bug bash on trunk.

I am attaching a file with payloads and logs which might help you.

  • Sharing settings will sync only if you change any metadata of that particular element along with change in sharing settings.
  • When we create Custom attributes and make them mandatory and assign to a data set (for example) and sync them to local instance, we get a run time exception stating that attribute for dataset not given any value. This is because even if it is mandatory we are not changing the dataset to set the value to attribute.
  • Data set - when updated with a data approval workflow - does not sync to local. Everything syncs and association is not syncing
  • when we updated the existing program with adding combination of categories and synched.
  • while updating attributes in the program with registration.
  • ERROR 2016-06-21 21:23:24,325 ERROR: duplicate key value violates unique constraint ““programtrackedentityattribute_unique_key””

Detail: Key (programid, trackedentityattributeid)=(255, 240) already exists. (SqlExceptionHelper.java [taskScheduler-6])

  • ERROR 2016-06-21 21:23:24,385 Exception occurred while trying to import the metadata. could not execute statement (MetadataImportHandler.java [taskScheduler-6])

org.hibernate.exception.ConstraintViolationException: could not execute statement at org.hibernate.exception.internal.SQLStateConversionDelegate.convert(SQLStateConversionDelegate.java:112)"Error on Edit Program Stage page. Press cancel and exception is thrown. Analytics Export Error. Process failed. null.

  • Charts, pivot tables, Report Table/ Pivot Table, Event Report, Event Chart, Validation Rule

Metadata Sync failed for the above metadata With the below runtime exception.

— Exception occurred while trying to import the metadata. object references an unsaved transient instance - save the transient instance before flushing: org.hisp.dhis.dataelement.DataElement (MetadataSyncTask.java [taskScheduler-9])

  • Admin user when added to dashboard item is not syncing with metadat sync.
  • When deleted a legend at HQ and synced to local. The legend at Local also is deleted on UI (exists in DB). Also, legend is getting duplicated with a different UID in DB.
  • Metadata Filter (Import-Export --> Metadata Detailed Export)

modification to properties like name, description is successfully synced but the selection is not synced, the payload also have the details of the updated data when synced.

  • custom forms are synced properly when created, but while Updation of custom form not reflecting on UI. However, payload has the full information of updates.

Thanks & Regards,

Nalinikanth M

Quality Analyst

Email
nalinim@thoughtworks.com
Telephone
+91 9052234588
ThoughtWorks

Alex Tumwesigye

Technical Advisor - DHIS2 (Consultant),
Ministry of Health/AFENET | HISP Uganda

Kampala

Uganda
+256 774149 775, + 256 759 800161

Skype ID: talexie

IT Consultant (Servers, Networks and Security, Health Information Systems - DHIS2, Disease Outbreak & Surveillance Systems) & Solar Consultant

"I don’t want to be anything other than what I have been - one tree hill "

Hi Alex

Just to be clear, you are testing on /api/23/metadata or /api/24/metadata endpoints? TW testing does not apply to legacy importer (available on /api/metadata)

···

On Fri, Jun 24, 2016 at 7:15 PM, Alex Tumwesigye atumwesigye@gmail.com wrote:

Dear Morten,

Yes, I have the same issues but I wanted to clearly document them as a bug. This happens when you extract the program metadata and then try to import it to an existing similar program (same uid but in production).

The following fails to be imported correctly

  • Option values are not imported in process
  • message reminder templates are not imported
  • programTrackedEntityAttributes are not imported (seems they are imported before attributes or viceversa, need to confirm this)
  • data entry stage custom forms are not imported

Will document well and submit the bug (after testing against the new release).

Thanks.

Alex

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Fri, Jun 24, 2016 at 2:32 PM, Morten Olav Hansen morten@dhis2.org wrote:

Hi

I will try and look into these issues, please be aware that I have done a lot of fixes to the importer in the last 2 days, so if this is from before that… probably you need to re-test.

Will go over them on Monday and see what can be fixed for release.


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp


Alex Tumwesigye

Technical Advisor - DHIS2 (Consultant),
Ministry of Health/AFENET | HISP Uganda

Kampala

Uganda
+256 774149 775, + 256 759 800161

Skype ID: talexie

IT Consultant (Servers, Networks and Security, Health Information Systems - DHIS2, Disease Outbreak & Surveillance Systems) & Solar Consultant

"I don’t want to be anything other than what I have been - one tree hill "

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Thu, Jun 23, 2016 at 7:28 PM, Nalinikanth Meesala nalinim@thoughtworks.com wrote:

Hi Lars & Morten,

I am working for MSF-OCA project from ThoughtWorks. As you are aware that we are building a metadata synchronisation. We had a bug bash recently and below are the bugs we found in new importer. We did the bug bash on trunk.

I am attaching a file with payloads and logs which might help you.

  • Sharing settings will sync only if you change any metadata of that particular element along with change in sharing settings.
  • When we create Custom attributes and make them mandatory and assign to a data set (for example) and sync them to local instance, we get a run time exception stating that attribute for dataset not given any value. This is because even if it is mandatory we are not changing the dataset to set the value to attribute.
  • Data set - when updated with a data approval workflow - does not sync to local. Everything syncs and association is not syncing
  • when we updated the existing program with adding combination of categories and synched.
  • while updating attributes in the program with registration.
  • ERROR 2016-06-21 21:23:24,325 ERROR: duplicate key value violates unique constraint ““programtrackedentityattribute_unique_key””

Detail: Key (programid, trackedentityattributeid)=(255, 240) already exists. (SqlExceptionHelper.java [taskScheduler-6])

  • ERROR 2016-06-21 21:23:24,385 Exception occurred while trying to import the metadata. could not execute statement (MetadataImportHandler.java [taskScheduler-6])

org.hibernate.exception.ConstraintViolationException: could not execute statement at org.hibernate.exception.internal.SQLStateConversionDelegate.convert(SQLStateConversionDelegate.java:112)"Error on Edit Program Stage page. Press cancel and exception is thrown. Analytics Export Error. Process failed. null.

  • Charts, pivot tables, Report Table/ Pivot Table, Event Report, Event Chart, Validation Rule

Metadata Sync failed for the above metadata With the below runtime exception.

— Exception occurred while trying to import the metadata. object references an unsaved transient instance - save the transient instance before flushing: org.hisp.dhis.dataelement.DataElement (MetadataSyncTask.java [taskScheduler-9])

  • Admin user when added to dashboard item is not syncing with metadat sync.
  • When deleted a legend at HQ and synced to local. The legend at Local also is deleted on UI (exists in DB). Also, legend is getting duplicated with a different UID in DB.
  • Metadata Filter (Import-Export --> Metadata Detailed Export)

modification to properties like name, description is successfully synced but the selection is not synced, the payload also have the details of the updated data when synced.

  • custom forms are synced properly when created, but while Updation of custom form not reflecting on UI. However, payload has the full information of updates.

Thanks & Regards,

Nalinikanth M

Quality Analyst

Email
nalinim@thoughtworks.com
Telephone
+91 9052234588
ThoughtWorks