importing datasets between 2 2.25 instances

Hi,

I am trying to import a dataset from a development instance (on 2.25) over to a production instance of 2.25. I exported the dataset from the development instance using the meta-data dependency export. I tried testing importing it locally (both through UI and by posting via cUrl) and am receiving the following error in the log file:

Caused by: org.postgresql.util.PSQLException: ERROR: null value in column “categorycomboid” violates not-null constraint

Detail: Failing row contains (363716, DTI66gwkiYc, PR_4, 2016-09-25 16:37:50.541, 2017-02-25 12:34:41.404, PR_MSM reached with HIV prevention {Site (Outreach, DIC)} (V1), PR_MSM reached, null, 1.5. Number of men who have sex with men reached with HIV preven…, INTEGER_ZERO_OR_POSITIVE, AGGREGATE, SUM, null, null, f, null, null, 176738, --------, null).

When I check the corresponding DE’s in the XML file, they all have categorycomboid’s (file attached). Is there anything else that might be causing this error?

Thanks,

Nick

pr v1.xml (257 KB)

Hi Nick

I’m looking into it (I can reproduce it locally)

···

On Sat, Feb 25, 2017 at 2:05 PM, Shurajit Dutta shurajitdutta@gmail.com wrote:

Hi,

I am trying to import a dataset from a development instance (on 2.25) over to a production instance of 2.25. I exported the dataset from the development instance using the meta-data dependency export. I tried testing importing it locally (both through UI and by posting via cUrl) and am receiving the following error in the log file:

Caused by: org.postgresql.util.PSQLException: ERROR: null value in column “categorycomboid” violates not-null constraint

Detail: Failing row contains (363716, DTI66gwkiYc, PR_4, 2016-09-25 16:37:50.541, 2017-02-25 12:34:41.404, PR_MSM reached with HIV prevention {Site (Outreach, DIC)} (V1), PR_MSM reached, null, 1.5. Number of men who have sex with men reached with HIV preven…, INTEGER_ZERO_OR_POSITIVE, AGGREGATE, SUM, null, null, f, null, null, 176738, --------, null).

When I check the corresponding DE’s in the XML file, they all have categorycomboid’s (file attached). Is there anything else that might be causing this error?

Thanks,

Nick


Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

Hi Nick

When did you update this instance last time? would you mind upgrading to latest 2.25 and trying again? You might still see similar errors, but I see from your payload… that the export is probably from an older version of 2.25?

Both instances should also run the same version of 2.25 (if possible, as there has been some backports)

···

On Tue, Feb 28, 2017 at 1:29 PM, Morten Olav Hansen morten@dhis2.org wrote:

Hi Nick

I’m looking into it (I can reproduce it locally)

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Sat, Feb 25, 2017 at 2:05 PM, Shurajit Dutta shurajitdutta@gmail.com wrote:

Hi,

I am trying to import a dataset from a development instance (on 2.25) over to a production instance of 2.25. I exported the dataset from the development instance using the meta-data dependency export. I tried testing importing it locally (both through UI and by posting via cUrl) and am receiving the following error in the log file:

Caused by: org.postgresql.util.PSQLException: ERROR: null value in column “categorycomboid” violates not-null constraint

Detail: Failing row contains (363716, DTI66gwkiYc, PR_4, 2016-09-25 16:37:50.541, 2017-02-25 12:34:41.404, PR_MSM reached with HIV prevention {Site (Outreach, DIC)} (V1), PR_MSM reached, null, 1.5. Number of men who have sex with men reached with HIV preven…, INTEGER_ZERO_OR_POSITIVE, AGGREGATE, SUM, null, null, f, null, null, 176738, --------, null).

When I check the corresponding DE’s in the XML file, they all have categorycomboid’s (file attached). Is there anything else that might be causing this error?

Thanks,

Nick


Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp

Hi Morten,
We upgraded to the absolute latest version (as of yesterday) and had the same problem. However, in this case, there was more to the story. In the destination system, a catcombo existed with exactly the same categories and category options, but with a different UID. After removing this in the destination system and reimporting the category combo, it worked.

What did not work however was using the payload from the metadata dependency exporter. I had to create a script to first extract

  1. All dependent category combos and cat option combos

  2. All dependent indicators

  3. All dependent data entry forms

  4. and 2) were uploaded prior to uploading the dataset. 3) was uploaded separately after the import of the dataset.

Neither 1 nor 2 worked via the UI, but only worked by POSTing direclty to the /api/25 end point.

So, in general, quite a bit of gymnastics. I really hope we can get the dependency exporter back in place ,as this has really complicated workflows where we are creating things on a dev instance and then moving it to a production instance.

Regards,

Jason

···

On Tue, Feb 28, 2017 at 9:44 AM, Morten Olav Hansen morten@dhis2.org wrote:

Hi Nick

When did you update this instance last time? would you mind upgrading to latest 2.25 and trying again? You might still see similar errors, but I see from your payload… that the export is probably from an older version of 2.25?

Both instances should also run the same version of 2.25 (if possible, as there has been some backports)


Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Tue, Feb 28, 2017 at 1:29 PM, Morten Olav Hansen morten@dhis2.org wrote:

Hi Nick

I’m looking into it (I can reproduce it locally)

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Sat, Feb 25, 2017 at 2:05 PM, Shurajit Dutta shurajitdutta@gmail.com wrote:

Hi,

I am trying to import a dataset from a development instance (on 2.25) over to a production instance of 2.25. I exported the dataset from the development instance using the meta-data dependency export. I tried testing importing it locally (both through UI and by posting via cUrl) and am receiving the following error in the log file:

Caused by: org.postgresql.util.PSQLException: ERROR: null value in column “categorycomboid” violates not-null constraint

Detail: Failing row contains (363716, DTI66gwkiYc, PR_4, 2016-09-25 16:37:50.541, 2017-02-25 12:34:41.404, PR_MSM reached with HIV prevention {Site (Outreach, DIC)} (V1), PR_MSM reached, null, 1.5. Number of men who have sex with men reached with HIV preven…, INTEGER_ZERO_OR_POSITIVE, AGGREGATE, SUM, null, null, f, null, null, 176738, --------, null).

When I check the corresponding DE’s in the XML file, they all have categorycomboid’s (file attached). Is there anything else that might be causing this error?

Thanks,

Nick


Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp

Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+46764147049

Jason,

Would you mind giving me a fresh XML dump of the data set metadata? the version Nick sent me, seem to be from an older version… I just want to try and see if it can be imported into an empty database

···

On Tue, Feb 28, 2017 at 4:09 PM, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Morten,
We upgraded to the absolute latest version (as of yesterday) and had the same problem. However, in this case, there was more to the story. In the destination system, a catcombo existed with exactly the same categories and category options, but with a different UID. After removing this in the destination system and reimporting the category combo, it worked.

What did not work however was using the payload from the metadata dependency exporter. I had to create a script to first extract

  1. All dependent category combos and cat option combos
  1. All dependent indicators
  1. All dependent data entry forms
  1. and 2) were uploaded prior to uploading the dataset. 3) was uploaded separately after the import of the dataset.

Neither 1 nor 2 worked via the UI, but only worked by POSTing direclty to the /api/25 end point.

So, in general, quite a bit of gymnastics. I really hope we can get the dependency exporter back in place ,as this has really complicated workflows where we are creating things on a dev instance and then moving it to a production instance.

Regards,

Jason

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Tue, Feb 28, 2017 at 9:44 AM, Morten Olav Hansen morten@dhis2.org wrote:

Hi Nick

When did you update this instance last time? would you mind upgrading to latest 2.25 and trying again? You might still see similar errors, but I see from your payload… that the export is probably from an older version of 2.25?

Both instances should also run the same version of 2.25 (if possible, as there has been some backports)


Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp

Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+46764147049

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Tue, Feb 28, 2017 at 1:29 PM, Morten Olav Hansen morten@dhis2.org wrote:

Hi Nick

I’m looking into it (I can reproduce it locally)

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Sat, Feb 25, 2017 at 2:05 PM, Shurajit Dutta shurajitdutta@gmail.com wrote:

Hi,

I am trying to import a dataset from a development instance (on 2.25) over to a production instance of 2.25. I exported the dataset from the development instance using the meta-data dependency export. I tried testing importing it locally (both through UI and by posting via cUrl) and am receiving the following error in the log file:

Caused by: org.postgresql.util.PSQLException: ERROR: null value in column “categorycomboid” violates not-null constraint

Detail: Failing row contains (363716, DTI66gwkiYc, PR_4, 2016-09-25 16:37:50.541, 2017-02-25 12:34:41.404, PR_MSM reached with HIV prevention {Site (Outreach, DIC)} (V1), PR_MSM reached, null, 1.5. Number of men who have sex with men reached with HIV preven…, INTEGER_ZERO_OR_POSITIVE, AGGREGATE, SUM, null, null, f, null, null, 176738, --------, null).

When I check the corresponding DE’s in the XML file, they all have categorycomboid’s (file attached). Is there anything else that might be causing this error?

Thanks,

Nick


Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp

Hi @Shurajit_Dutta and @jason.

I am willing to export dataset data from our partner organization’s DHIS 2 instance and import it into my organization’s DHIS 2 instance.

I am seeking a step-by-step guidance on how to do this. I would like to know the key steps to achieve this. How will mapping of the exported data be done in our instance?

Many thanks