Data element not found or not accessible

Dear,

I get the message “Data element not found or not accessible” when trying to import a csv file. I looked at some old threads, but failed to find an answer.

The csv file is attached (with headers, but it gave the same answer without headers).

Attached also the API page of the data element, the sharing of the data element (I have superuser access), and the orgunit page to confirm the ID. Also a screenshot of the import summary and my DHIS2 instance info.

I guess I might be missing something but I can’t figure out what exactly.

Thank you in advance for your help.

Kind regards,

Thomas Lavreys

Thomas Lavreys, MSc, PgC Trop Med

Sustainability Department SOCFINCO

Maisha Consulting bvba

Bisschoppenhof 9

3150 Tildonk

Belgium

Tel Socfin: +32-25-080050

Tel Maisha: +32-16-850242

Cell phone: +32-474-717641

Fax: +32-16-850242

TEST.csv (628 Bytes)

image

image

image

image

image

Hi Thomas,

Can you try specifying the catoptioncombo even if it is default? This was an issue with one of the version builds a while ago.

Let’s know if that helps.

Cheers

- -

Busoye Anifalaje (PhD)
Director of Services (Principal), BAO Systems
UK: +44 7901-740-757 | US: +1 682-307-0986|
busoye@baosystems.com <mailto:busoye@baosystems.com> | http://www.baosystems.com <http://www.baosystems.com/&gt; |
Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007

···

On 17 Jan 2018, at 12:02, Thomas Lavreys <thomaslavreys@gmail.com> wrote:

Dear,

I get the message “Data element not found or not accessible” when trying to import a csv file. I looked at some old threads, but failed to find an answer.
The csv file is attached (with headers, but it gave the same answer without headers).
Attached also the API page of the data element, the sharing of the data element (I have superuser access), and the orgunit page to confirm the ID. Also a screenshot of the import summary and my DHIS2 instance info.

I guess I might be missing something but I can’t figure out what exactly.

Thank you in advance for your help.

Kind regards,

Thomas Lavreys

Thomas Lavreys, MSc, PgC Trop Med
Sustainability Department SOCFINCO
Maisha Consulting bvba
Bisschoppenhof 9
3150 Tildonk
Belgium

Tel Socfin: +32-25-080050
Tel Maisha: +32-16-850242
Cell phone: +32-474-717641
Fax: +32-16-850242

<TEST.csv><data element id.JPG><sharing data element.JPG><orgunit id.JPG><import csv.JPG><instance info.JPG>_______________________________________________
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net <mailto:dhis2-users@lists.launchpad.net>
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp

Hi Busoye,

I only seem to be able to find the category combo id ("categoryCombo":{"id":"bjDvmb4bfuf"}). This is default.
Is this the same as the catoptioncombo?

Regards,

Thomas

image

···

From: Adebusoye Anifalaje [mailto:busoye@baosystems.com]
Sent: Wednesday, January 17, 2018 12:11 PM
To: Thomas Lavreys thomaslavreys@gmail.com
Cc: dhis2-users@lists.launchpad.net
Subject: Re: [Dhis2-users] Data element not found or not accessible

Hi Thomas,

Can you try specifying the catoptioncombo even if it is default? This was an issue with one of the version builds a while ago.

Let’s know if that helps.

Cheers

Busoye Anifalaje (PhD)
Director of Services (Principal), BAO Systems

UK: +44 7901-740-757 | US: +1 682-307-0986|

busoye@baosystems.com | http://www.baosystems.com |

Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007

On 17 Jan 2018, at 12:02, Thomas Lavreys thomaslavreys@gmail.com wrote:

Dear,

I get the message “Data element not found or not accessible” when trying to import a csv file. I looked at some old threads, but failed to find an answer.

The csv file is attached (with headers, but it gave the same answer without headers).

Attached also the API page of the data element, the sharing of the data element (I have superuser access), and the orgunit page to confirm the ID. Also a screenshot of the import summary and my DHIS2 instance info.

I guess I might be missing something but I can’t figure out what exactly.

Thank you in advance for your help.

Kind regards,

Thomas Lavreys

Thomas Lavreys, MSc, PgC Trop Med

Sustainability Department SOCFINCO

Maisha Consulting bvba

Bisschoppenhof 9

3150 Tildonk

Belgium

**

Tel Socfin: +32-25-080050

Tel Maisha: +32-16-850242

Cell phone: +32-474-717641

Fax: +32-16-850242

<TEST.csv>_______________________________________________
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp

I tried it anyway, and it doesn’t recognize it:

Import summary

Import count

Type

Count

Imported

0

Updated

0

Deleted

0

Ignored

13

Conflicts

Element

Description

dataelement

Data element not found or not accessible

bjDvmb4bfuf

Category option combo not found or not accessible

So I’m guessing I need to find the catoptioncombo ID somewhere.

Regards,

Thomas

image

···

From: Adebusoye Anifalaje [mailto:busoye@baosystems.com]
Sent: Wednesday, January 17, 2018 12:11 PM
To: Thomas Lavreys thomaslavreys@gmail.com
Cc: dhis2-users@lists.launchpad.net
Subject: Re: [Dhis2-users] Data element not found or not accessible

Hi Thomas,

Can you try specifying the catoptioncombo even if it is default? This was an issue with one of the version builds a while ago.

Let’s know if that helps.

Cheers

Busoye Anifalaje (PhD)
Director of Services (Principal), BAO Systems

UK: +44 7901-740-757 | US: +1 682-307-0986|

busoye@baosystems.com | http://www.baosystems.com |

Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007

On 17 Jan 2018, at 12:02, Thomas Lavreys thomaslavreys@gmail.com wrote:

Dear,

I get the message “Data element not found or not accessible” when trying to import a csv file. I looked at some old threads, but failed to find an answer.

The csv file is attached (with headers, but it gave the same answer without headers).

Attached also the API page of the data element, the sharing of the data element (I have superuser access), and the orgunit page to confirm the ID. Also a screenshot of the import summary and my DHIS2 instance info.

I guess I might be missing something but I can’t figure out what exactly.

Thank you in advance for your help.

Kind regards,

Thomas Lavreys

Thomas Lavreys, MSc, PgC Trop Med

Sustainability Department SOCFINCO

Maisha Consulting bvba

Bisschoppenhof 9

3150 Tildonk

Belgium

**

Tel Socfin: +32-25-080050

Tel Maisha: +32-16-850242

Cell phone: +32-474-717641

Fax: +32-16-850242

<TEST.csv>_______________________________________________
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp

Hi Thomas,

This isn’t the same. One way to do this from the front end is to search apps for Category Option Combination and then filter for default. Once you are here, you can show details to get the catoptioncombo uid

See screenshots below.

Cheers

- -

Busoye Anifalaje (PhD)
Director of Services (Principal), BAO Systems
UK: +44 7901-740-757 | US: +1 682-307-0986|
busoye@baosystems.com <mailto:busoye@baosystems.com> | http://www.baosystems.com <http://www.baosystems.com/&gt; |
Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007

···

On 17 Jan 2018, at 12:18, Thomas Lavreys <thomaslavreys@gmail.com> wrote:

Hi Busoye,

I only seem to be able to find the category combo id ("categoryCombo":{"id":"bjDvmb4bfuf"}). This is default.
Is this the same as the catoptioncombo?

Regards,

Thomas
<>
From: Adebusoye Anifalaje [mailto:busoye@baosystems.com <mailto:busoye@baosystems.com>]
Sent: Wednesday, January 17, 2018 12:11 PM
To: Thomas Lavreys <thomaslavreys@gmail.com <mailto:thomaslavreys@gmail.com>>
Cc: dhis2-users@lists.launchpad.net <mailto:dhis2-users@lists.launchpad.net>
Subject: Re: [Dhis2-users] Data element not found or not accessible

Hi Thomas,

Can you try specifying the catoptioncombo even if it is default? This was an issue with one of the version builds a while ago.

Let’s know if that helps.

Cheers

- -

<image001.png>

Busoye Anifalaje (PhD)
Director of Services (Principal), BAO Systems
UK: +44 7901-740-757 | US: +1 682-307-0986|
busoye@baosystems.com <mailto:busoye@baosystems.com> | http://www.baosystems.com <http://www.baosystems.com/&gt; |
Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007

On 17 Jan 2018, at 12:02, Thomas Lavreys <thomaslavreys@gmail.com <mailto:thomaslavreys@gmail.com>> wrote:

Dear,

I get the message “Data element not found or not accessible” when trying to import a csv file. I looked at some old threads, but failed to find an answer.
The csv file is attached (with headers, but it gave the same answer without headers).
Attached also the API page of the data element, the sharing of the data element (I have superuser access), and the orgunit page to confirm the ID. Also a screenshot of the import summary and my DHIS2 instance info.

I guess I might be missing something but I can’t figure out what exactly.

Thank you in advance for your help.

Kind regards,

Thomas Lavreys

Thomas Lavreys, MSc, PgC Trop Med
Sustainability Department SOCFINCO
Maisha Consulting bvba
Bisschoppenhof 9
3150 Tildonk
Belgium

Tel Socfin: +32-25-080050
Tel Maisha: +32-16-850242
Cell phone: +32-474-717641
Fax: +32-16-850242

<TEST.csv><data element id.JPG><sharing data element.JPG><orgunit id.JPG><import csv.JPG><instance info.JPG>_______________________________________________
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net <mailto:dhis2-users@lists.launchpad.net>
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp

Hi Thomas,

I aslo see that you are using version 2.23, it is possible that the csv column structure is different for this version. There might be someone on this forum who is currently working on this version and can provide input on that.
- -

Busoye Anifalaje (PhD)
Director of Services (Principal), BAO Systems
UK: +44 7901-740-757 | US: +1 682-307-0986|
busoye@baosystems.com <mailto:busoye@baosystems.com> | http://www.baosystems.com <http://www.baosystems.com/&gt; |
Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007

···

On 17 Jan 2018, at 12:23, Thomas Lavreys <thomaslavreys@gmail.com> wrote:

I tried it anyway, and it doesn’t recognize it:

Import summary
Import count
Type
Count
Imported
0
Updated
0
Deleted
0
Ignored
13
Conflicts
Element
Description
dataelement
Data element not found or not accessible
bjDvmb4bfuf
Category option combo not found or not accessible

So I’m guessing I need to find the catoptioncombo ID somewhere. <>

Regards,

Thomas

From: Adebusoye Anifalaje [mailto:busoye@baosystems.com <mailto:busoye@baosystems.com>]
Sent: Wednesday, January 17, 2018 12:11 PM
To: Thomas Lavreys <thomaslavreys@gmail.com <mailto:thomaslavreys@gmail.com>>
Cc: dhis2-users@lists.launchpad.net <mailto:dhis2-users@lists.launchpad.net>
Subject: Re: [Dhis2-users] Data element not found or not accessible

Hi Thomas,

Can you try specifying the catoptioncombo even if it is default? This was an issue with one of the version builds a while ago.

Let’s know if that helps.

Cheers

- -

<image001.png>

Busoye Anifalaje (PhD)
Director of Services (Principal), BAO Systems
UK: +44 7901-740-757 | US: +1 682-307-0986|
busoye@baosystems.com <mailto:busoye@baosystems.com> | http://www.baosystems.com <http://www.baosystems.com/&gt; |
Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007

On 17 Jan 2018, at 12:02, Thomas Lavreys <thomaslavreys@gmail.com <mailto:thomaslavreys@gmail.com>> wrote:

Dear,

I get the message “Data element not found or not accessible” when trying to import a csv file. I looked at some old threads, but failed to find an answer.
The csv file is attached (with headers, but it gave the same answer without headers).
Attached also the API page of the data element, the sharing of the data element (I have superuser access), and the orgunit page to confirm the ID. Also a screenshot of the import summary and my DHIS2 instance info.

I guess I might be missing something but I can’t figure out what exactly.

Thank you in advance for your help.

Kind regards,

Thomas Lavreys

Thomas Lavreys, MSc, PgC Trop Med
Sustainability Department SOCFINCO
Maisha Consulting bvba
Bisschoppenhof 9
3150 Tildonk
Belgium

Tel Socfin: +32-25-080050
Tel Maisha: +32-16-850242
Cell phone: +32-474-717641
Fax: +32-16-850242

<TEST.csv><data element id.JPG><sharing data element.JPG><orgunit id.JPG><import csv.JPG><instance info.JPG>_______________________________________________
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net <mailto:dhis2-users@lists.launchpad.net>
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp

I don’t seem to have a ‘default’ category option combo:

I’ll have a look at how to get it there.

Kind regards,

Thomas

image

image

image

···

From: Adebusoye Anifalaje [mailto:busoye@baosystems.com]
Sent: Wednesday, January 17, 2018 12:26 PM
To: Thomas Lavreys thomaslavreys@gmail.com
Cc: dhis2-users@lists.launchpad.net
Subject: Re: [Dhis2-users] Data element not found or not accessible

Hi Thomas,

This isn’t the same. One way to do this from the front end is to search apps for Category Option Combination and then filter for default. Once you are here, you can show details to get the catoptioncombo uid

See screenshots below.

Cheers

Busoye Anifalaje (PhD)
Director of Services (Principal), BAO Systems

UK: +44 7901-740-757 | US: +1 682-307-0986|

busoye@baosystems.com | http://www.baosystems.com |

Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007

On 17 Jan 2018, at 12:18, Thomas Lavreys thomaslavreys@gmail.com wrote:

Hi Busoye,

I only seem to be able to find the category combo id ("categoryCombo":{"id":"bjDvmb4bfuf"}). This is default.
Is this the same as the catoptioncombo?
 
Regards,
 
Thomas

From: Adebusoye Anifalaje [mailto:busoye@baosystems.com]
Sent: Wednesday, January 17, 2018 12:11 PM
To: Thomas Lavreys thomaslavreys@gmail.com
Cc: dhis2-users@lists.launchpad.net
Subject: Re: [Dhis2-users] Data element not found or not accessible

Hi Thomas,

Can you try specifying the catoptioncombo even if it is default? This was an issue with one of the version builds a while ago.

Let’s know if that helps.

Cheers

**
<image001.png>

**

Busoye Anifalaje (PhD)
Director of Services (Principal), BAO Systems

UK: +44 7901-740-757 | US: +1 682-307-0986|

busoye@baosystems.com | http://www.baosystems.com |

Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007

On 17 Jan 2018, at 12:02, Thomas Lavreys thomaslavreys@gmail.com wrote:

Dear,

I get the message “Data element not found or not accessible” when trying to import a csv file. I looked at some old threads, but failed to find an answer.

The csv file is attached (with headers, but it gave the same answer without headers).

Attached also the API page of the data element, the sharing of the data element (I have superuser access), and the orgunit page to confirm the ID. Also a screenshot of the import summary and my DHIS2 instance info.

I guess I might be missing something but I can’t figure out what exactly.

Thank you in advance for your help.

Kind regards,

Thomas Lavreys

Thomas Lavreys, MSc, PgC Trop Med

Sustainability Department SOCFINCO

Maisha Consulting bvba

Bisschoppenhof 9

3150 Tildonk

Belgium

**

Tel Socfin: +32-25-080050

Tel Maisha: +32-16-850242

Cell phone: +32-474-717641

Fax: +32-16-850242

<TEST.csv>_______________________________________________
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp

I found the issue. The file was saved as a csv utf-8, which is different to a normal csv (aparrently).

Thanks for the advice!

Kind regards,

Thomas

image

···

From: Adebusoye Anifalaje [mailto:busoye@baosystems.com]
Sent: Wednesday, January 17, 2018 12:31 PM
To: Thomas Lavreys thomaslavreys@gmail.com
Cc: dhis2-users@lists.launchpad.net
Subject: Re: [Dhis2-users] Data element not found or not accessible

Hi Thomas,

I aslo see that you are using version 2.23, it is possible that the csv column structure is different for this version. There might be someone on this forum who is currently working on this version and can provide input on that.

Busoye Anifalaje (PhD)
Director of Services (Principal), BAO Systems

UK: +44 7901-740-757 | US: +1 682-307-0986|

busoye@baosystems.com | http://www.baosystems.com |

Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007

On 17 Jan 2018, at 12:23, Thomas Lavreys thomaslavreys@gmail.com wrote:

I tried it anyway, and it doesn’t recognize it:

Import summary

Import count

Type

Count

Imported

0

Updated

0

Deleted

0

Ignored

13

Conflicts

Element

Description

dataelement

Data element not found or not accessible

bjDvmb4bfuf

Category option combo not found or not accessible

So I’m guessing I need to find the catoptioncombo ID somewhere.

Regards,

Thomas

From: Adebusoye Anifalaje [mailto:busoye@baosystems.com]
Sent: Wednesday, January 17, 2018 12:11 PM
To: Thomas Lavreys thomaslavreys@gmail.com
Cc: dhis2-users@lists.launchpad.net
Subject: Re: [Dhis2-users] Data element not found or not accessible

Hi Thomas,

Can you try specifying the catoptioncombo even if it is default? This was an issue with one of the version builds a while ago.

Let’s know if that helps.

Cheers

**
<image001.png>

**

Busoye Anifalaje (PhD)
Director of Services (Principal), BAO Systems

UK: +44 7901-740-757 | US: +1 682-307-0986|

busoye@baosystems.com | http://www.baosystems.com |

Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007

On 17 Jan 2018, at 12:02, Thomas Lavreys thomaslavreys@gmail.com wrote:

Dear,

I get the message “Data element not found or not accessible” when trying to import a csv file. I looked at some old threads, but failed to find an answer.

The csv file is attached (with headers, but it gave the same answer without headers).

Attached also the API page of the data element, the sharing of the data element (I have superuser access), and the orgunit page to confirm the ID. Also a screenshot of the import summary and my DHIS2 instance info.

I guess I might be missing something but I can’t figure out what exactly.

Thank you in advance for your help.

Kind regards,

Thomas Lavreys

Thomas Lavreys, MSc, PgC Trop Med

Sustainability Department SOCFINCO

Maisha Consulting bvba

Bisschoppenhof 9

3150 Tildonk

Belgium

**

Tel Socfin: +32-25-080050

Tel Maisha: +32-16-850242

Cell phone: +32-474-717641

Fax: +32-16-850242

<TEST.csv>_______________________________________________
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp

Hello Thomas!

I am glad you have solved your issue! I noticed a few gotchas so here are a couple of things to note when importing csv files in DHIS 2:

  1. The first row

dataelement Data element not found or not accessible

image

is a ** funny error**, because it is actually telling you something about a single row, found here in the documentation:

"dataelement","period","orgunit","catoptcombo","attroptcombo","value","storedby","lastupd","comment"

DHIS 2 is literally complaining about the fact that it cannot find a data element with the uid “dataelement”, which is what is assumes you are trying to load in that top row. You can try yourself by loading a file with only this top line, the error will be the same, 1 ignored row, data element not found or not accessible. It is safe to ignore this error, I’d be worried if there was a UID called dataelement, which is actually a valid UID! Just extremely unlikely to pop up from the random UID generator.

  1. Ignored

Unlike the metadata importer, the data importer will report that it ignored every row on a dry run. It can make testing difficult because you don’t know if it will actually import correctly or not as it reports back that every row was ignored. Two rules of thumb here:

  1. The actual errored out row will be listed in the “Conflicts” section, even in a dry run

  2. Sadly, some errors you can only find on a real run, so always upload your files with a comment, so you can find them later via the DB to delete them if something goes wrong and all else fails

  3. File Type

UTF-8 works well and I use it for all my CSV uploads. Be wary of EXCEL’s CSV saving habits though, especially on a Mac. Your file should be UTF-8 without a BOM (a byte order marker, DHIS 2 doesn’t really care, as it will error out on row one anyway (see above)) AND it should be standard unix line feeds (LF). Excel for the Mac will default to saving it with circa 1990’s CRs only AND a BOM (unless you are editing a CSV, in which case it will keep the settings it found when it opened it). Easiest way to check is to grab a copy of Notepad++ (windows) or BBedit (mac).

BBedit:

image

and

image

Notepad++

image

and

image

Hope this helps!

image

···

On Wed, Jan 17, 2018 at 9:11 AM, Thomas Lavreys thomaslavreys@gmail.com wrote:

I found the issue. The file was saved as a csv utf-8, which is different to a normal csv (aparrently).

Thanks for the advice!

Kind regards,

Thomas

From: Adebusoye Anifalaje [mailto:busoye@baosystems.com]
Sent: Wednesday, January 17, 2018 12:31 PM

To: Thomas Lavreys thomaslavreys@gmail.com
Cc: dhis2-users@lists.launchpad.net
Subject: Re: [Dhis2-users] Data element not found or not accessible

Hi Thomas,

I aslo see that you are using version 2.23, it is possible that the csv column structure is different for this version. There might be someone on this forum who is currently working on this version and can provide input on that.

Busoye Anifalaje (PhD)
Director of Services (Principal), BAO Systems

UK: +44 7901-740-757 | US: +1 682-307-0986|

busoye@baosystems.com | http://www.baosystems.com |

Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007

On 17 Jan 2018, at 12:23, Thomas Lavreys thomaslavreys@gmail.com wrote:

I tried it anyway, and it doesn’t recognize it:

Import summary

Import count

Type

Count

Imported

0

Updated

0

Deleted

0

Ignored

13

Conflicts

Element

Description

dataelement

Data element not found or not accessible

bjDvmb4bfuf

Category option combo not found or not accessible

So I’m guessing I need to find the catoptioncombo ID somewhere.

Regards,

Thomas

From: Adebusoye Anifalaje [mailto:busoye@baosystems.com]
Sent: Wednesday, January 17, 2018 12:11 PM
To: Thomas Lavreys thomaslavreys@gmail.com
Cc: dhis2-users@lists.launchpad.net
Subject: Re: [Dhis2-users] Data element not found or not accessible

Hi Thomas,

Can you try specifying the catoptioncombo even if it is default? This was an issue with one of the version builds a while ago.

Let’s know if that helps.

Cheers

**
<image001.png>

**

Busoye Anifalaje (PhD)
Director of Services (Principal), BAO Systems

UK: +44 7901-740-757 | US: +1 682-307-0986|

busoye@baosystems.com | http://www.baosystems.com |

Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007

On 17 Jan 2018, at 12:02, Thomas Lavreys thomaslavreys@gmail.com wrote:

Dear,

I get the message “Data element not found or not accessible” when trying to import a csv file. I looked at some old threads, but failed to find an answer.

The csv file is attached (with headers, but it gave the same answer without headers).

Attached also the API page of the data element, the sharing of the data element (I have superuser access), and the orgunit page to confirm the ID. Also a screenshot of the import summary and my DHIS2 instance info.

I guess I might be missing something but I can’t figure out what exactly.

Thank you in advance for your help.

Kind regards,

Thomas Lavreys

Thomas Lavreys, MSc, PgC Trop Med

Sustainability Department SOCFINCO

Maisha Consulting bvba

Bisschoppenhof 9

3150 Tildonk

Belgium

**

Tel Socfin: +32-25-080050

Tel Maisha: +32-16-850242

Cell phone: +32-474-717641

Fax: +32-16-850242

<TEST.csv>_______________________________________________
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp


Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp

Timothy Harding
Sr. Systems Analyst, BAO Systems

+1 202-536-1541 | tharding@baosystems.com | http://www.baosystems.com | Skype: hardingt@gmail.com | 2900 K Street, Suite 406, Washington D.C. 20007

Thanks for the in depth info! It will surely help along the way. I’ve only started with some testers, but I will probably run into some more issues when I load my real data.

Thanks again!!

Kind regards,

Thomas

image

image

image

image

image

image

···

From: Timothy Harding [mailto:tharding@baosystems.com]
Sent: Wednesday, January 17, 2018 3:35 PM
To: Thomas Lavreys thomaslavreys@gmail.com
Cc: Adebusoye Anifalaje busoye@baosystems.com; dhis2-users dhis2-users@lists.launchpad.net
Subject: Re: [Dhis2-users] Data element not found or not accessible

Hello Thomas!

I am glad you have solved your issue! I noticed a few gotchas so here are a couple of things to note when importing csv files in DHIS 2:

  1. The first row

dataelement Data element not found or not accessible

Inline image 1

is a ** funny error**, because it is actually telling you something about a single row, found here in the documentation:

"dataelement","period","orgunit","catoptcombo","attroptcombo","value","storedby","lastupd","comment"

DHIS 2 is literally complaining about the fact that it cannot find a data element with the uid “dataelement”, which is what is assumes you are trying to load in that top row. You can try yourself by loading a file with only this top line, the error will be the same, 1 ignored row, data element not found or not accessible. It is safe to ignore this error, I’d be worried if there was a UID called dataelement, which is actually a valid UID! Just extremely unlikely to pop up from the random UID generator.

  1. Ignored

Unlike the metadata importer, the data importer will report that it ignored every row on a dry run. It can make testing difficult because you don’t know if it will actually import correctly or not as it reports back that every row was ignored. Two rules of thumb here:

  1. The actual errored out row will be listed in the “Conflicts” section, even in a dry run

  2. Sadly, some errors you can only find on a real run, so always upload your files with a comment, so you can find them later via the DB to delete them if something goes wrong and all else fails

  3. File Type

UTF-8 works well and I use it for all my CSV uploads. Be wary of EXCEL’s CSV saving habits though, especially on a Mac. Your file should be UTF-8 without a BOM (a byte order marker, DHIS 2 doesn’t really care, as it will error out on row one anyway (see above)) AND it should be standard unix line feeds (LF). Excel for the Mac will default to saving it with circa 1990’s CRs only AND a BOM (unless you are editing a CSV, in which case it will keep the settings it found when it opened it). Easiest way to check is to grab a copy of Notepad++ (windows) or BBedit (mac).

BBedit:

Inline image 2

and

Inline image 3

Notepad++

Inline image 4

and

Inline image 5

Hope this helps!

Timothy Harding
Sr. Systems Analyst, BAO Systems

+1 202-536-1541 | tharding@baosystems.com | http://www.baosystems.com | Skype: hardingt@gmail.com | 2900 K Street, Suite 406, Washington D.C. 20007

On Wed, Jan 17, 2018 at 9:11 AM, Thomas Lavreys thomaslavreys@gmail.com wrote:

I found the issue. The file was saved as a csv utf-8, which is different to a normal csv (aparrently).

Thanks for the advice!

Kind regards,

Thomas

From: Adebusoye Anifalaje [mailto:busoye@baosystems.com]
Sent: Wednesday, January 17, 2018 12:31 PM

To: Thomas Lavreys thomaslavreys@gmail.com
Cc: dhis2-users@lists.launchpad.net
Subject: Re: [Dhis2-users] Data element not found or not accessible

Hi Thomas,

I aslo see that you are using version 2.23, it is possible that the csv column structure is different for this version. There might be someone on this forum who is currently working on this version and can provide input on that.

Busoye Anifalaje (PhD)
Director of Services (Principal), BAO Systems

UK: +44 7901-740-757 | US: +1 682-307-0986|

busoye@baosystems.com | http://www.baosystems.com |

Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007

On 17 Jan 2018, at 12:23, Thomas Lavreys thomaslavreys@gmail.com wrote:

I tried it anyway, and it doesn’t recognize it:

Import summary

Import count

Type

Count

Imported

0

Updated

0

Deleted

0

Ignored

13

Conflicts

Element

Description

dataelement

Data element not found or not accessible

bjDvmb4bfuf

Category option combo not found or not accessible

So I’m guessing I need to find the catoptioncombo ID somewhere.

Regards,

Thomas

From: Adebusoye Anifalaje [mailto:busoye@baosystems.com]
Sent: Wednesday, January 17, 2018 12:11 PM
To: Thomas Lavreys thomaslavreys@gmail.com
Cc: dhis2-users@lists.launchpad.net
Subject: Re: [Dhis2-users] Data element not found or not accessible

Hi Thomas,

Can you try specifying the catoptioncombo even if it is default? This was an issue with one of the version builds a while ago.

Let’s know if that helps.

Cheers

**
<image001.png>
**

Busoye Anifalaje (PhD)
Director of Services (Principal), BAO Systems

UK: +44 7901-740-757 | US: +1 682-307-0986|

busoye@baosystems.com | http://www.baosystems.com |

Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007

On 17 Jan 2018, at 12:02, Thomas Lavreys thomaslavreys@gmail.com wrote:

Dear,

I get the message “Data element not found or not accessible” when trying to import a csv file. I looked at some old threads, but failed to find an answer.

The csv file is attached (with headers, but it gave the same answer without headers).

Attached also the API page of the data element, the sharing of the data element (I have superuser access), and the orgunit page to confirm the ID. Also a screenshot of the import summary and my DHIS2 instance info.

I guess I might be missing something but I can’t figure out what exactly.

Thank you in advance for your help.

Kind regards,

Thomas Lavreys

Thomas Lavreys, MSc, PgC Trop Med

Sustainability Department SOCFINCO

Maisha Consulting bvba

Bisschoppenhof 9

3150 Tildonk

Belgium

**

Tel Socfin: +32-25-080050

Tel Maisha: +32-16-850242

Cell phone: +32-474-717641

Fax: +32-16-850242

<TEST.csv>_______________________________________________
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp


Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp

Hi all,

Thanks Timothy for sharing this. I am trying to import organisation units
metadata in csv file with UTF-8 encoding. When I import the file with
Metadata import in dhis2, the file is not imported at all and I do get the
summary report.

I wrote the csv file with R function write.table.

Its funny enough that when I write the same file using Excel and save as
csv file with UTF-8, it works well.

I do not understand what could be the problem here, has anyone tried to
write a csv file with an R function that can successfully be imported in
the dhis2?

Looking for your help. I have attached my csv files in this email.

Thanks

···

==========================
Isaiah Nyabuto
+254729572616
Bsc. Health Records Management and Informatics

On Jan 17, 2018 17:51, "Thomas Lavreys" <thomaslavreys@gmail.com> wrote:

Thanks for the in depth info! It will surely help along the way. I’ve only
started with some testers, but I will probably run into some more issues
when I load my real data.

Thanks again!!

Kind regards,

Thomas

*From:* Timothy Harding [mailto:tharding@baosystems.com]
*Sent:* Wednesday, January 17, 2018 3:35 PM
*To:* Thomas Lavreys <thomaslavreys@gmail.com>
*Cc:* Adebusoye Anifalaje <busoye@baosystems.com>; dhis2-users <
dhis2-users@lists.launchpad.net>
*Subject:* Re: [Dhis2-users] Data element not found or not accessible

Hello Thomas!

I am glad you have solved your issue! I noticed a few gotchas so here are
a couple of things to note when importing csv files in DHIS 2:

1. The first row

dataelement Data element not found or not accessible

[image: Inline image 1]

is a* funny error*, because it is actually telling you something about a
single row, found here in the documentation
<Home - DHIS2 Documentation;
:

"dataelement","period","orgunit","catoptcombo","attroptcombo","value","storedby","lastupd","comment"

DHIS 2 is literally complaining about the fact that it *cannot find a
data element with the uid "dataelement"*, which is what is assumes you
are trying to load in that top row. You can try yourself by loading a file
with only this top line, the error will be the same, 1 ignored row, data
element not found or not accessible. It is safe to ignore this error, I'd
be worried if there was a UID called dataelement, which is actually a valid
UID! Just extremely unlikely to pop up from the random UID generator.

2. Ignored

Unlike the metadata importer, the data importer will report that it
ignored *every row on a dry run*. It can make testing difficult because
you don't know if it will actually import correctly or not as it reports
back that every row was ignored. Two rules of thumb here:

   1. The actual errored out row will be listed in the "Conflicts"
   section, even in a dry run
   2. Sadly, some errors you can only find on a real run, so always
   upload your files with a comment, so you can find them later via the DB to
   delete them if something goes wrong and all else fails

3. File Type

UTF-8 works well and I use it for all my CSV uploads. Be wary of EXCEL's
CSV saving habits though, especially on a Mac. Your file should be *UTF-8
without a BOM* (a byte order marker, DHIS 2 doesn't _really_ care, as it
will error out on row one anyway (see above)) AND it should be standard *unix
line feeds (LF)*. Excel for the Mac will default to saving it with circa
1990's CRs only AND a BOM (unless you are editing a CSV, in which case it
will keep the settings it found when it opened it). Easiest way to check is
to grab a copy of Notepad++ (windows) or BBedit (mac).

BBedit:

[image: Inline image 2]

and

[image: Inline image 3]

Notepad++

[image: Inline image 4]

and

[image: Inline image 5]

Hope this helps!

*Timothy Harding*
Sr. Systems Analyst, BAO Systems

+1 202-536-1541 | tharding@baosystems.com | http://www.baosystems.com | Skype:
hardingt@gmail.com | 2900 K Street, Suite 406, Washington D.C. 20007

On Wed, Jan 17, 2018 at 9:11 AM, Thomas Lavreys <thomaslavreys@gmail.com> > wrote:

I found the issue. The file was saved as a csv utf-8, which is different
to a normal csv (aparrently).

Thanks for the advice!

Kind regards,

Thomas

*From:* Adebusoye Anifalaje [mailto:busoye@baosystems.com]
*Sent:* Wednesday, January 17, 2018 12:31 PM

*To:* Thomas Lavreys <thomaslavreys@gmail.com>
*Cc:* dhis2-users@lists.launchpad.net
*Subject:* Re: [Dhis2-users] Data element not found or not accessible

Hi Thomas,

I aslo see that you are using version 2.23, it is possible that the csv
column structure is different for this version. There might be someone on
this forum who is currently working on this version and can provide input
on that.

- -

*Busoye Anifalaje (PhD)*
Director of Services (Principal), BAO Systems

UK: +44 7901-740-757 <+44%207901%20740757> | US: +1 682-307-0986
<(682)%20307-0986>|

busoye@baosystems.com | http://www.baosystems.com |

Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007

On 17 Jan 2018, at 12:23, Thomas Lavreys <thomaslavreys@gmail.com> wrote:

I tried it anyway, and it doesn’t recognize it:

Import summary

Import count

*Type*

*Count*

Imported

0

Updated

0

Deleted

0

Ignored

13

Conflicts

*Element*

*Description*

dataelement

Data element not found or not accessible

bjDvmb4bfuf

Category option combo not found or not accessible

So I’m guessing I need to find the catoptioncombo ID somewhere.

Regards,

Thomas

*From:* Adebusoye Anifalaje [mailto:busoye@baosystems.com
<busoye@baosystems.com>]
*Sent:* Wednesday, January 17, 2018 12:11 PM
*To:* Thomas Lavreys <thomaslavreys@gmail.com>
*Cc:* dhis2-users@lists.launchpad.net
*Subject:* Re: [Dhis2-users] Data element not found or not accessible

Hi Thomas,

Can you try specifying the catoptioncombo even if it is default? This was
an issue with one of the version builds a while ago.

Let’s know if that helps.

Cheers

- -

*<image001.png>*

*Busoye Anifalaje (PhD)*
Director of Services (Principal), BAO Systems

UK: +44 7901-740-757 <+44%207901%20740757> | US: +1 682-307-0986
<(682)%20307-0986>|

busoye@baosystems.com | http://www.baosystems.com |

Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007

On 17 Jan 2018, at 12:02, Thomas Lavreys <thomaslavreys@gmail.com> wrote:

Dear,

I get the message “Data element not found or not accessible” when trying
to import a csv file. I looked at some old threads, but failed to find an
answer.

The csv file is attached (with headers, but it gave the same answer
without headers).

Attached also the API page of the data element, the sharing of the data
element (I have superuser access), and the orgunit page to confirm the ID.
Also a screenshot of the import summary and my DHIS2 instance info.

I guess I might be missing something but I can’t figure out what exactly.

Thank you in advance for your help.

Kind regards,

Thomas Lavreys

*Thomas Lavreys, MSc, PgC Trop Med*

*Sustainability Department SOCFINCO*

*Maisha Consulting bvba*

*Bisschoppenhof 9*

*3150 Tildonk*

*Belgium*

*Tel Socfin: +32-25-080050*

*Tel Maisha: +32-16-850242 <+32%2016%2085%2002%2042>*

*Cell phone: +32-474-717641 <+32%20474%2071%2076%2041>*

*Fax: +32-16-850242 <+32%2016%2085%2002%2042>*

<TEST.csv><data element id.JPG><sharing data element.JPG><orgunit
id.JPG><import csv.JPG><instance info.JPG>_____________________
__________________________
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp

_______________________________________________
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp

_______________________________________________
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp

Isaiah:

org_kqmh_list_test.csv:

Has an extra column

image

1, 2, 3, etc… should not be in there unless the OUs are named 1, 2, 3.

Probably should be LF instead of CRLF:

image

but I’m not sure it makes a difference here.

Also you’ve got a lot of NA’s in there, not sure those are needed unless you are setting something to be NA, best just have , rather than ,NA,

image

Good luck!

image

image

image

image

image

image

···

On Mon, Jan 29, 2018 at 11:30 AM, Isaiah Nyabuto isaiahnyabuto@gmail.com wrote:

Hi all,

Thanks Timothy for sharing this. I am trying to import organisation units metadata in csv file with UTF-8 encoding. When I import the file with Metadata import in dhis2, the file is not imported at all and I do get the summary report.

I wrote the csv file with R function write.table.

Its funny enough that when I write the same file using Excel and save as csv file with UTF-8, it works well.

I do not understand what could be the problem here, has anyone tried to write a csv file with an R function that can successfully be imported in the dhis2?

Looking for your help. I have attached my csv files in this email.

Thanks

==========================
Isaiah Nyabuto
+254729572616
Bsc. Health Records Management and Informatics

Timothy Harding
Sr. Systems Analyst, BAO Systems

+1 202-536-1541 | tharding@baosystems.com | http://www.baosystems.com | Skype: hardingt@gmail.com | 2900 K Street, Suite 406, Washington D.C. 20007

On Jan 17, 2018 17:51, “Thomas Lavreys” thomaslavreys@gmail.com wrote:

Thanks for the in depth info! It will surely help along the way. I’ve only started with some testers, but I will probably run into some more issues when I load my real data.

Thanks again!!

Kind regards,

Thomas

From: Timothy Harding [mailto:tharding@baosystems.com]
Sent: Wednesday, January 17, 2018 3:35 PM
To: Thomas Lavreys thomaslavreys@gmail.com
Cc: Adebusoye Anifalaje busoye@baosystems.com; dhis2-users dhis2-users@lists.launchpad.net
Subject: Re: [Dhis2-users] Data element not found or not accessible

Hello Thomas!

I am glad you have solved your issue! I noticed a few gotchas so here are a couple of things to note when importing csv files in DHIS 2:

  1. The first row

dataelement Data element not found or not accessible

is a ** funny error**, because it is actually telling you something about a single row, found here in the documentation:

"dataelement","period","orgunit","catoptcombo","attroptcombo","value","storedby","lastupd","comment"

DHIS 2 is literally complaining about the fact that it cannot find a data element with the uid “dataelement”, which is what is assumes you are trying to load in that top row. You can try yourself by loading a file with only this top line, the error will be the same, 1 ignored row, data element not found or not accessible. It is safe to ignore this error, I’d be worried if there was a UID called dataelement, which is actually a valid UID! Just extremely unlikely to pop up from the random UID generator.

  1. Ignored

Unlike the metadata importer, the data importer will report that it ignored every row on a dry run. It can make testing difficult because you don’t know if it will actually import correctly or not as it reports back that every row was ignored. Two rules of thumb here:

  1. The actual errored out row will be listed in the “Conflicts” section, even in a dry run
  2. Sadly, some errors you can only find on a real run, so always upload your files with a comment, so you can find them later via the DB to delete them if something goes wrong and all else fails
  1. File Type

UTF-8 works well and I use it for all my CSV uploads. Be wary of EXCEL’s CSV saving habits though, especially on a Mac. Your file should be UTF-8 without a BOM (a byte order marker, DHIS 2 doesn’t really care, as it will error out on row one anyway (see above)) AND it should be standard unix line feeds (LF). Excel for the Mac will default to saving it with circa 1990’s CRs only AND a BOM (unless you are editing a CSV, in which case it will keep the settings it found when it opened it). Easiest way to check is to grab a copy of Notepad++ (windows) or BBedit (mac).

BBedit:

and

Notepad++

and

Hope this helps!

Timothy Harding
Sr. Systems Analyst, BAO Systems

+1 202-536-1541 | tharding@baosystems.com | http://www.baosystems.com | Skype: hardingt@gmail.com | 2900 K Street, Suite 406, Washington D.C. 20007

On Wed, Jan 17, 2018 at 9:11 AM, Thomas Lavreys thomaslavreys@gmail.com wrote:

I found the issue. The file was saved as a csv utf-8, which is different to a normal csv (aparrently).

Thanks for the advice!

Kind regards,

Thomas

From: Adebusoye Anifalaje [mailto:busoye@baosystems.com]
Sent: Wednesday, January 17, 2018 12:31 PM

To: Thomas Lavreys thomaslavreys@gmail.com
Cc: dhis2-users@lists.launchpad.net
Subject: Re: [Dhis2-users] Data element not found or not accessible

Hi Thomas,

I aslo see that you are using version 2.23, it is possible that the csv column structure is different for this version. There might be someone on this forum who is currently working on this version and can provide input on that.

Busoye Anifalaje (PhD)
Director of Services (Principal), BAO Systems

UK: +44 7901-740-757 | US: +1 682-307-0986|

busoye@baosystems.com | http://www.baosystems.com |

Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007

On 17 Jan 2018, at 12:23, Thomas Lavreys thomaslavreys@gmail.com wrote:

I tried it anyway, and it doesn’t recognize it:

Import summary

Import count

Type

Count

Imported

0

Updated

0

Deleted

0

Ignored

13

Conflicts

Element

Description

dataelement

Data element not found or not accessible

bjDvmb4bfuf

Category option combo not found or not accessible

So I’m guessing I need to find the catoptioncombo ID somewhere.

Regards,

Thomas

From: Adebusoye Anifalaje [mailto:busoye@baosystems.com]
Sent: Wednesday, January 17, 2018 12:11 PM
To: Thomas Lavreys thomaslavreys@gmail.com
Cc: dhis2-users@lists.launchpad.net
Subject: Re: [Dhis2-users] Data element not found or not accessible

Hi Thomas,

Can you try specifying the catoptioncombo even if it is default? This was an issue with one of the version builds a while ago.

Let’s know if that helps.

Cheers

**
<image001.png>
**

Busoye Anifalaje (PhD)
Director of Services (Principal), BAO Systems

UK: +44 7901-740-757 | US: +1 682-307-0986|

busoye@baosystems.com | http://www.baosystems.com |

Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007

On 17 Jan 2018, at 12:02, Thomas Lavreys thomaslavreys@gmail.com wrote:

Dear,

I get the message “Data element not found or not accessible” when trying to import a csv file. I looked at some old threads, but failed to find an answer.

The csv file is attached (with headers, but it gave the same answer without headers).

Attached also the API page of the data element, the sharing of the data element (I have superuser access), and the orgunit page to confirm the ID. Also a screenshot of the import summary and my DHIS2 instance info.

I guess I might be missing something but I can’t figure out what exactly.

Thank you in advance for your help.

Kind regards,

Thomas Lavreys

Thomas Lavreys, MSc, PgC Trop Med

Sustainability Department SOCFINCO

Maisha Consulting bvba

Bisschoppenhof 9

3150 Tildonk

Belgium

**

Tel Socfin: +32-25-080050

Tel Maisha: +32-16-850242

Cell phone: +32-474-717641

Fax: +32-16-850242

<TEST.csv>_______________________________________________
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp


Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp


Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp

Hi Isaiah,

Yes, R is definitely capable of creating a file which you can import into DHIS2.

If you want to use CSV, be sure to have your file with the "row.names=FALSE " parameter and be sure to convert all “NA” values to with " na = ‘’ "

You can also post the file directly to DHIS2. Have a look at https://github.com/jason-p-pickering/dhis2-data-munging/blob/master/bootstrap/bootstrap.Rmd or more details of how to do this in the section “Loading the data elements”

Regards,

Jason

image

image

image

image

image

image

image

image

image

···

On Mon, Jan 29, 2018 at 6:04 PM, Timothy Harding tharding@baosystems.com wrote:

Isaiah:

org_kqmh_list_test.csv:

Has an extra column

1, 2, 3, etc… should not be in there unless the OUs are named 1, 2, 3.

Probably should be LF instead of CRLF:

but I’m not sure it makes a difference here.

Also you’ve got a lot of NA’s in there, not sure those are needed unless you are setting something to be NA, best just have , rather than ,NA,

Good luck!

Timothy Harding
Sr. Systems Analyst, BAO Systems

+1 202-536-1541 | tharding@baosystems.com | http://www.baosystems.com | Skype: hardingt@gmail.com | 2900 K Street, Suite 406, Washington D.C. 20007

On Mon, Jan 29, 2018 at 11:30 AM, Isaiah Nyabuto isaiahnyabuto@gmail.com wrote:

Hi all,

Thanks Timothy for sharing this. I am trying to import organisation units metadata in csv file with UTF-8 encoding. When I import the file with Metadata import in dhis2, the file is not imported at all and I do get the summary report.

I wrote the csv file with R function write.table.

Its funny enough that when I write the same file using Excel and save as csv file with UTF-8, it works well.

I do not understand what could be the problem here, has anyone tried to write a csv file with an R function that can successfully be imported in the dhis2?

Looking for your help. I have attached my csv files in this email.

Thanks

==========================
Isaiah Nyabuto
+254729572616
Bsc. Health Records Management and Informatics

On Jan 17, 2018 17:51, “Thomas Lavreys” thomaslavreys@gmail.com wrote:

Thanks for the in depth info! It will surely help along the way. I’ve only started with some testers, but I will probably run into some more issues when I load my real data.

Thanks again!!

Kind regards,

Thomas

From: Timothy Harding [mailto:tharding@baosystems.com]
Sent: Wednesday, January 17, 2018 3:35 PM
To: Thomas Lavreys thomaslavreys@gmail.com
Cc: Adebusoye Anifalaje busoye@baosystems.com; dhis2-users dhis2-users@lists.launchpad.net
Subject: Re: [Dhis2-users] Data element not found or not accessible

Hello Thomas!

I am glad you have solved your issue! I noticed a few gotchas so here are a couple of things to note when importing csv files in DHIS 2:

  1. The first row

dataelement Data element not found or not accessible

is a ** funny error**, because it is actually telling you something about a single row, found here in the documentation:

"dataelement","period","orgunit","catoptcombo","attroptcombo","value","storedby","lastupd","comment"

DHIS 2 is literally complaining about the fact that it cannot find a data element with the uid “dataelement”, which is what is assumes you are trying to load in that top row. You can try yourself by loading a file with only this top line, the error will be the same, 1 ignored row, data element not found or not accessible. It is safe to ignore this error, I’d be worried if there was a UID called dataelement, which is actually a valid UID! Just extremely unlikely to pop up from the random UID generator.

  1. Ignored

Unlike the metadata importer, the data importer will report that it ignored every row on a dry run. It can make testing difficult because you don’t know if it will actually import correctly or not as it reports back that every row was ignored. Two rules of thumb here:

  1. The actual errored out row will be listed in the “Conflicts” section, even in a dry run
  2. Sadly, some errors you can only find on a real run, so always upload your files with a comment, so you can find them later via the DB to delete them if something goes wrong and all else fails
  1. File Type

UTF-8 works well and I use it for all my CSV uploads. Be wary of EXCEL’s CSV saving habits though, especially on a Mac. Your file should be UTF-8 without a BOM (a byte order marker, DHIS 2 doesn’t really care, as it will error out on row one anyway (see above)) AND it should be standard unix line feeds (LF). Excel for the Mac will default to saving it with circa 1990’s CRs only AND a BOM (unless you are editing a CSV, in which case it will keep the settings it found when it opened it). Easiest way to check is to grab a copy of Notepad++ (windows) or BBedit (mac).

BBedit:

and

Notepad++

and

Hope this helps!

Timothy Harding
Sr. Systems Analyst, BAO Systems

+1 202-536-1541 | tharding@baosystems.com | http://www.baosystems.com | Skype: hardingt@gmail.com | 2900 K Street, Suite 406, Washington D.C. 20007

On Wed, Jan 17, 2018 at 9:11 AM, Thomas Lavreys thomaslavreys@gmail.com wrote:

I found the issue. The file was saved as a csv utf-8, which is different to a normal csv (aparrently).

Thanks for the advice!

Kind regards,

Thomas

From: Adebusoye Anifalaje [mailto:busoye@baosystems.com]
Sent: Wednesday, January 17, 2018 12:31 PM

To: Thomas Lavreys thomaslavreys@gmail.com
Cc: dhis2-users@lists.launchpad.net
Subject: Re: [Dhis2-users] Data element not found or not accessible

Hi Thomas,

I aslo see that you are using version 2.23, it is possible that the csv column structure is different for this version. There might be someone on this forum who is currently working on this version and can provide input on that.

Busoye Anifalaje (PhD)
Director of Services (Principal), BAO Systems

UK: +44 7901-740-757 | US: +1 682-307-0986|

busoye@baosystems.com | http://www.baosystems.com |

Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007

On 17 Jan 2018, at 12:23, Thomas Lavreys thomaslavreys@gmail.com wrote:

I tried it anyway, and it doesn’t recognize it:

Import summary

Import count

Type

Count

Imported

0

Updated

0

Deleted

0

Ignored

13

Conflicts

Element

Description

dataelement

Data element not found or not accessible

bjDvmb4bfuf

Category option combo not found or not accessible

So I’m guessing I need to find the catoptioncombo ID somewhere.

Regards,

Thomas

From: Adebusoye Anifalaje [mailto:busoye@baosystems.com]
Sent: Wednesday, January 17, 2018 12:11 PM
To: Thomas Lavreys thomaslavreys@gmail.com
Cc: dhis2-users@lists.launchpad.net
Subject: Re: [Dhis2-users] Data element not found or not accessible

Hi Thomas,

Can you try specifying the catoptioncombo even if it is default? This was an issue with one of the version builds a while ago.

Let’s know if that helps.

Cheers

**
<image001.png>
**

Busoye Anifalaje (PhD)
Director of Services (Principal), BAO Systems

UK: +44 7901-740-757 | US: +1 682-307-0986|

busoye@baosystems.com | http://www.baosystems.com |

Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007

On 17 Jan 2018, at 12:02, Thomas Lavreys thomaslavreys@gmail.com wrote:

Dear,

I get the message “Data element not found or not accessible” when trying to import a csv file. I looked at some old threads, but failed to find an answer.

The csv file is attached (with headers, but it gave the same answer without headers).

Attached also the API page of the data element, the sharing of the data element (I have superuser access), and the orgunit page to confirm the ID. Also a screenshot of the import summary and my DHIS2 instance info.

I guess I might be missing something but I can’t figure out what exactly.

Thank you in advance for your help.

Kind regards,

Thomas Lavreys

Thomas Lavreys, MSc, PgC Trop Med

Sustainability Department SOCFINCO

Maisha Consulting bvba

Bisschoppenhof 9

3150 Tildonk

Belgium

**

Tel Socfin: +32-25-080050

Tel Maisha: +32-16-850242

Cell phone: +32-474-717641

Fax: +32-16-850242

<TEST.csv>_______________________________________________
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp


Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp


Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp

Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+46764147049

Isaiah,

I’ve found the best way is to convert rows of a CSV into JSON structure and then post to the appropriate endpoint using httr.

I’ve made a slew of helper functions that make it easy for me to do along with some markdown files starting to explain things. When I started working with dhis2, I was an analyst and most comfortable using dataframes, so this kind of fits that perspective. I’ve learned a lot along the way and plan to refactor things at some point. Eventually, I’d like to make it a real R package- just haven’t had the time!

https://github.com/aaroncwhite/dhis2R

Feel free to PM me for more detail/explanation.

Aaron

image

image

image

image

image

image

image

image

image

···

On Mon, Jan 29, 2018 at 12:24 PM, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Isaiah,

Yes, R is definitely capable of creating a file which you can import into DHIS2.

If you want to use CSV, be sure to have your file with the "row.names=FALSE " parameter and be sure to convert all “NA” values to with " na = ‘’ "

You can also post the file directly to DHIS2. Have a look at https://github.com/jason-p-pickering/dhis2-data-munging/blob/master/bootstrap/bootstrap.Rmd or more details of how to do this in the section “Loading the data elements”

Regards,

Jason


Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp

Aaron C. White

919.522.5585

On Mon, Jan 29, 2018 at 6:04 PM, Timothy Harding tharding@baosystems.com wrote:

Isaiah:

org_kqmh_list_test.csv:

Has an extra column

1, 2, 3, etc… should not be in there unless the OUs are named 1, 2, 3.

Probably should be LF instead of CRLF:

but I’m not sure it makes a difference here.

Also you’ve got a lot of NA’s in there, not sure those are needed unless you are setting something to be NA, best just have , rather than ,NA,

Good luck!

Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+46764147049

Timothy Harding
Sr. Systems Analyst, BAO Systems

+1 202-536-1541 | tharding@baosystems.com | http://www.baosystems.com | Skype: hardingt@gmail.com | 2900 K Street, Suite 406, Washington D.C. 20007

On Mon, Jan 29, 2018 at 11:30 AM, Isaiah Nyabuto isaiahnyabuto@gmail.com wrote:

Hi all,

Thanks Timothy for sharing this. I am trying to import organisation units metadata in csv file with UTF-8 encoding. When I import the file with Metadata import in dhis2, the file is not imported at all and I do get the summary report.

I wrote the csv file with R function write.table.

Its funny enough that when I write the same file using Excel and save as csv file with UTF-8, it works well.

I do not understand what could be the problem here, has anyone tried to write a csv file with an R function that can successfully be imported in the dhis2?

Looking for your help. I have attached my csv files in this email.

Thanks

==========================
Isaiah Nyabuto
+254729572616
Bsc. Health Records Management and Informatics

On Jan 17, 2018 17:51, “Thomas Lavreys” thomaslavreys@gmail.com wrote:

Thanks for the in depth info! It will surely help along the way. I’ve only started with some testers, but I will probably run into some more issues when I load my real data.

Thanks again!!

Kind regards,

Thomas

From: Timothy Harding [mailto:tharding@baosystems.com]
Sent: Wednesday, January 17, 2018 3:35 PM
To: Thomas Lavreys thomaslavreys@gmail.com
Cc: Adebusoye Anifalaje busoye@baosystems.com; dhis2-users dhis2-users@lists.launchpad.net
Subject: Re: [Dhis2-users] Data element not found or not accessible

Hello Thomas!

I am glad you have solved your issue! I noticed a few gotchas so here are a couple of things to note when importing csv files in DHIS 2:

  1. The first row

dataelement Data element not found or not accessible

is a ** funny error**, because it is actually telling you something about a single row, found here in the documentation:

"dataelement","period","orgunit","catoptcombo","attroptcombo","value","storedby","lastupd","comment"

DHIS 2 is literally complaining about the fact that it cannot find a data element with the uid “dataelement”, which is what is assumes you are trying to load in that top row. You can try yourself by loading a file with only this top line, the error will be the same, 1 ignored row, data element not found or not accessible. It is safe to ignore this error, I’d be worried if there was a UID called dataelement, which is actually a valid UID! Just extremely unlikely to pop up from the random UID generator.

  1. Ignored

Unlike the metadata importer, the data importer will report that it ignored every row on a dry run. It can make testing difficult because you don’t know if it will actually import correctly or not as it reports back that every row was ignored. Two rules of thumb here:

  1. The actual errored out row will be listed in the “Conflicts” section, even in a dry run
  2. Sadly, some errors you can only find on a real run, so always upload your files with a comment, so you can find them later via the DB to delete them if something goes wrong and all else fails
  1. File Type

UTF-8 works well and I use it for all my CSV uploads. Be wary of EXCEL’s CSV saving habits though, especially on a Mac. Your file should be UTF-8 without a BOM (a byte order marker, DHIS 2 doesn’t really care, as it will error out on row one anyway (see above)) AND it should be standard unix line feeds (LF). Excel for the Mac will default to saving it with circa 1990’s CRs only AND a BOM (unless you are editing a CSV, in which case it will keep the settings it found when it opened it). Easiest way to check is to grab a copy of Notepad++ (windows) or BBedit (mac).

BBedit:

and

Notepad++

and

Hope this helps!

Timothy Harding
Sr. Systems Analyst, BAO Systems

+1 202-536-1541 | tharding@baosystems.com | http://www.baosystems.com | Skype: hardingt@gmail.com | 2900 K Street, Suite 406, Washington D.C. 20007

On Wed, Jan 17, 2018 at 9:11 AM, Thomas Lavreys thomaslavreys@gmail.com wrote:

I found the issue. The file was saved as a csv utf-8, which is different to a normal csv (aparrently).

Thanks for the advice!

Kind regards,

Thomas

From: Adebusoye Anifalaje [mailto:busoye@baosystems.com]
Sent: Wednesday, January 17, 2018 12:31 PM

To: Thomas Lavreys thomaslavreys@gmail.com
Cc: dhis2-users@lists.launchpad.net
Subject: Re: [Dhis2-users] Data element not found or not accessible

Hi Thomas,

I aslo see that you are using version 2.23, it is possible that the csv column structure is different for this version. There might be someone on this forum who is currently working on this version and can provide input on that.

Busoye Anifalaje (PhD)
Director of Services (Principal), BAO Systems

UK: +44 7901-740-757 | US: +1 682-307-0986|

busoye@baosystems.com | http://www.baosystems.com |

Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007

On 17 Jan 2018, at 12:23, Thomas Lavreys thomaslavreys@gmail.com wrote:

I tried it anyway, and it doesn’t recognize it:

Import summary

Import count

Type

Count

Imported

0

Updated

0

Deleted

0

Ignored

13

Conflicts

Element

Description

dataelement

Data element not found or not accessible

bjDvmb4bfuf

Category option combo not found or not accessible

So I’m guessing I need to find the catoptioncombo ID somewhere.

Regards,

Thomas

From: Adebusoye Anifalaje [mailto:busoye@baosystems.com]
Sent: Wednesday, January 17, 2018 12:11 PM
To: Thomas Lavreys thomaslavreys@gmail.com
Cc: dhis2-users@lists.launchpad.net
Subject: Re: [Dhis2-users] Data element not found or not accessible

Hi Thomas,

Can you try specifying the catoptioncombo even if it is default? This was an issue with one of the version builds a while ago.

Let’s know if that helps.

Cheers

**
<image001.png>
**

Busoye Anifalaje (PhD)
Director of Services (Principal), BAO Systems

UK: +44 7901-740-757 | US: +1 682-307-0986|

busoye@baosystems.com | http://www.baosystems.com |

Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007

On 17 Jan 2018, at 12:02, Thomas Lavreys thomaslavreys@gmail.com wrote:

Dear,

I get the message “Data element not found or not accessible” when trying to import a csv file. I looked at some old threads, but failed to find an answer.

The csv file is attached (with headers, but it gave the same answer without headers).

Attached also the API page of the data element, the sharing of the data element (I have superuser access), and the orgunit page to confirm the ID. Also a screenshot of the import summary and my DHIS2 instance info.

I guess I might be missing something but I can’t figure out what exactly.

Thank you in advance for your help.

Kind regards,

Thomas Lavreys

Thomas Lavreys, MSc, PgC Trop Med

Sustainability Department SOCFINCO

Maisha Consulting bvba

Bisschoppenhof 9

3150 Tildonk

Belgium

**

Tel Socfin: +32-25-080050

Tel Maisha: +32-16-850242

Cell phone: +32-474-717641

Fax: +32-16-850242

<TEST.csv>_______________________________________________
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp


Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp


Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp

Hi Jason,

This is really helpful. Thanks for sharing the link too.

Cheers!

image

image

image

image

image

image

image

image

image

···

On Mon, Jan 29, 2018 at 8:24 PM, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Isaiah,

Yes, R is definitely capable of creating a file which you can import into DHIS2.

If you want to use CSV, be sure to have your file with the "row.names=FALSE " parameter and be sure to convert all “NA” values to with " na = ‘’ "

You can also post the file directly to DHIS2. Have a look at https://github.com/jason-p-pickering/dhis2-data-munging/blob/master/bootstrap/bootstrap.Rmd or more details of how to do this in the section “Loading the data elements”

Regards,

Jason

On Mon, Jan 29, 2018 at 6:04 PM, Timothy Harding tharding@baosystems.com wrote:

Isaiah:

org_kqmh_list_test.csv:

Has an extra column

1, 2, 3, etc… should not be in there unless the OUs are named 1, 2, 3.

Probably should be LF instead of CRLF:

but I’m not sure it makes a difference here.

Also you’ve got a lot of NA’s in there, not sure those are needed unless you are setting something to be NA, best just have , rather than ,NA,

Good luck!

Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+46764147049

Timothy Harding
Sr. Systems Analyst, BAO Systems

+1 202-536-1541 | tharding@baosystems.com | http://www.baosystems.com | Skype: hardingt@gmail.com | 2900 K Street, Suite 406, Washington D.C. 20007

On Mon, Jan 29, 2018 at 11:30 AM, Isaiah Nyabuto isaiahnyabuto@gmail.com wrote:

Hi all,

Thanks Timothy for sharing this. I am trying to import organisation units metadata in csv file with UTF-8 encoding. When I import the file with Metadata import in dhis2, the file is not imported at all and I do get the summary report.

I wrote the csv file with R function write.table.

Its funny enough that when I write the same file using Excel and save as csv file with UTF-8, it works well.

I do not understand what could be the problem here, has anyone tried to write a csv file with an R function that can successfully be imported in the dhis2?

Looking for your help. I have attached my csv files in this email.

Thanks

==========================
Isaiah Nyabuto
+254729572616
Bsc. Health Records Management and Informatics

On Jan 17, 2018 17:51, “Thomas Lavreys” thomaslavreys@gmail.com wrote:

Thanks for the in depth info! It will surely help along the way. I’ve only started with some testers, but I will probably run into some more issues when I load my real data.

Thanks again!!

Kind regards,

Thomas

From: Timothy Harding [mailto:tharding@baosystems.com]
Sent: Wednesday, January 17, 2018 3:35 PM
To: Thomas Lavreys thomaslavreys@gmail.com
Cc: Adebusoye Anifalaje busoye@baosystems.com; dhis2-users dhis2-users@lists.launchpad.net
Subject: Re: [Dhis2-users] Data element not found or not accessible

Hello Thomas!

I am glad you have solved your issue! I noticed a few gotchas so here are a couple of things to note when importing csv files in DHIS 2:

  1. The first row

dataelement Data element not found or not accessible

is a ** funny error**, because it is actually telling you something about a single row, found here in the documentation:

"dataelement","period","orgunit","catoptcombo","attroptcombo","value","storedby","lastupd","comment"

DHIS 2 is literally complaining about the fact that it cannot find a data element with the uid “dataelement”, which is what is assumes you are trying to load in that top row. You can try yourself by loading a file with only this top line, the error will be the same, 1 ignored row, data element not found or not accessible. It is safe to ignore this error, I’d be worried if there was a UID called dataelement, which is actually a valid UID! Just extremely unlikely to pop up from the random UID generator.

  1. Ignored

Unlike the metadata importer, the data importer will report that it ignored every row on a dry run. It can make testing difficult because you don’t know if it will actually import correctly or not as it reports back that every row was ignored. Two rules of thumb here:

  1. The actual errored out row will be listed in the “Conflicts” section, even in a dry run
  2. Sadly, some errors you can only find on a real run, so always upload your files with a comment, so you can find them later via the DB to delete them if something goes wrong and all else fails
  1. File Type

UTF-8 works well and I use it for all my CSV uploads. Be wary of EXCEL’s CSV saving habits though, especially on a Mac. Your file should be UTF-8 without a BOM (a byte order marker, DHIS 2 doesn’t really care, as it will error out on row one anyway (see above)) AND it should be standard unix line feeds (LF). Excel for the Mac will default to saving it with circa 1990’s CRs only AND a BOM (unless you are editing a CSV, in which case it will keep the settings it found when it opened it). Easiest way to check is to grab a copy of Notepad++ (windows) or BBedit (mac).

BBedit:

and

Notepad++

and

Hope this helps!

Timothy Harding
Sr. Systems Analyst, BAO Systems

+1 202-536-1541 | tharding@baosystems.com | http://www.baosystems.com | Skype: hardingt@gmail.com | 2900 K Street, Suite 406, Washington D.C. 20007

On Wed, Jan 17, 2018 at 9:11 AM, Thomas Lavreys thomaslavreys@gmail.com wrote:

I found the issue. The file was saved as a csv utf-8, which is different to a normal csv (aparrently).

Thanks for the advice!

Kind regards,

Thomas

From: Adebusoye Anifalaje [mailto:busoye@baosystems.com]
Sent: Wednesday, January 17, 2018 12:31 PM

To: Thomas Lavreys thomaslavreys@gmail.com
Cc: dhis2-users@lists.launchpad.net
Subject: Re: [Dhis2-users] Data element not found or not accessible

Hi Thomas,

I aslo see that you are using version 2.23, it is possible that the csv column structure is different for this version. There might be someone on this forum who is currently working on this version and can provide input on that.

Busoye Anifalaje (PhD)
Director of Services (Principal), BAO Systems

UK: +44 7901-740-757 | US: +1 682-307-0986|

busoye@baosystems.com | http://www.baosystems.com |

Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007

On 17 Jan 2018, at 12:23, Thomas Lavreys thomaslavreys@gmail.com wrote:

I tried it anyway, and it doesn’t recognize it:

Import summary

Import count

Type

Count

Imported

0

Updated

0

Deleted

0

Ignored

13

Conflicts

Element

Description

dataelement

Data element not found or not accessible

bjDvmb4bfuf

Category option combo not found or not accessible

So I’m guessing I need to find the catoptioncombo ID somewhere.

Regards,

Thomas

From: Adebusoye Anifalaje [mailto:busoye@baosystems.com]
Sent: Wednesday, January 17, 2018 12:11 PM
To: Thomas Lavreys thomaslavreys@gmail.com
Cc: dhis2-users@lists.launchpad.net
Subject: Re: [Dhis2-users] Data element not found or not accessible

Hi Thomas,

Can you try specifying the catoptioncombo even if it is default? This was an issue with one of the version builds a while ago.

Let’s know if that helps.

Cheers

**
<image001.png>
**

Busoye Anifalaje (PhD)
Director of Services (Principal), BAO Systems

UK: +44 7901-740-757 | US: +1 682-307-0986|

busoye@baosystems.com | http://www.baosystems.com |

Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007

On 17 Jan 2018, at 12:02, Thomas Lavreys thomaslavreys@gmail.com wrote:

Dear,

I get the message “Data element not found or not accessible” when trying to import a csv file. I looked at some old threads, but failed to find an answer.

The csv file is attached (with headers, but it gave the same answer without headers).

Attached also the API page of the data element, the sharing of the data element (I have superuser access), and the orgunit page to confirm the ID. Also a screenshot of the import summary and my DHIS2 instance info.

I guess I might be missing something but I can’t figure out what exactly.

Thank you in advance for your help.

Kind regards,

Thomas Lavreys

Thomas Lavreys, MSc, PgC Trop Med

Sustainability Department SOCFINCO

Maisha Consulting bvba

Bisschoppenhof 9

3150 Tildonk

Belgium

**

Tel Socfin: +32-25-080050

Tel Maisha: +32-16-850242

Cell phone: +32-474-717641

Fax: +32-16-850242

<TEST.csv>_______________________________________________
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp


Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp


Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp


Isaiah Nyabuto
+254 729 572 616

Bsc. Health Records Management & Informatics, JKUAT.

Hi Aron,

Thanks for sharing your scripts and its good to know we have the same Idea.

I often write several functions to access dhis2 APIs, I realized most of this could be standardized into an R package - R dhis2 wrapper. This is also something I foresee and I would like to write in the near future. If possible, may I ask we work together on that?

Thanks.

image

image

image

image

image

image

image

image

image

···

On Mon, Jan 29, 2018 at 9:00 PM, Aaron C White aaroncwhite@gmail.com wrote:

Isaiah,

I’ve found the best way is to convert rows of a CSV into JSON structure and then post to the appropriate endpoint using httr.

I’ve made a slew of helper functions that make it easy for me to do along with some markdown files starting to explain things. When I started working with dhis2, I was an analyst and most comfortable using dataframes, so this kind of fits that perspective. I’ve learned a lot along the way and plan to refactor things at some point. Eventually, I’d like to make it a real R package- just haven’t had the time!

https://github.com/aaroncwhite/dhis2R

Feel free to PM me for more detail/explanation.

Aaron


Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp

Aaron C. White

919.522.5585

On Mon, Jan 29, 2018 at 12:24 PM, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Isaiah,

Yes, R is definitely capable of creating a file which you can import into DHIS2.

If you want to use CSV, be sure to have your file with the "row.names=FALSE " parameter and be sure to convert all “NA” values to with " na = ‘’ "

You can also post the file directly to DHIS2. Have a look at https://github.com/jason-p-pickering/dhis2-data-munging/blob/master/bootstrap/bootstrap.Rmd or more details of how to do this in the section “Loading the data elements”

Regards,

Jason


Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp

On Mon, Jan 29, 2018 at 6:04 PM, Timothy Harding tharding@baosystems.com wrote:

Isaiah:

org_kqmh_list_test.csv:

Has an extra column

1, 2, 3, etc… should not be in there unless the OUs are named 1, 2, 3.

Probably should be LF instead of CRLF:

but I’m not sure it makes a difference here.

Also you’ve got a lot of NA’s in there, not sure those are needed unless you are setting something to be NA, best just have , rather than ,NA,

Good luck!

Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+46764147049

Timothy Harding
Sr. Systems Analyst, BAO Systems

+1 202-536-1541 | tharding@baosystems.com | http://www.baosystems.com | Skype: hardingt@gmail.com | 2900 K Street, Suite 406, Washington D.C. 20007

On Mon, Jan 29, 2018 at 11:30 AM, Isaiah Nyabuto isaiahnyabuto@gmail.com wrote:

Hi all,

Thanks Timothy for sharing this. I am trying to import organisation units metadata in csv file with UTF-8 encoding. When I import the file with Metadata import in dhis2, the file is not imported at all and I do get the summary report.

I wrote the csv file with R function write.table.

Its funny enough that when I write the same file using Excel and save as csv file with UTF-8, it works well.

I do not understand what could be the problem here, has anyone tried to write a csv file with an R function that can successfully be imported in the dhis2?

Looking for your help. I have attached my csv files in this email.

Thanks

==========================
Isaiah Nyabuto
+254729572616
Bsc. Health Records Management and Informatics

On Jan 17, 2018 17:51, “Thomas Lavreys” thomaslavreys@gmail.com wrote:

Thanks for the in depth info! It will surely help along the way. I’ve only started with some testers, but I will probably run into some more issues when I load my real data.

Thanks again!!

Kind regards,

Thomas

From: Timothy Harding [mailto:tharding@baosystems.com]
Sent: Wednesday, January 17, 2018 3:35 PM
To: Thomas Lavreys thomaslavreys@gmail.com
Cc: Adebusoye Anifalaje busoye@baosystems.com; dhis2-users dhis2-users@lists.launchpad.net
Subject: Re: [Dhis2-users] Data element not found or not accessible

Hello Thomas!

I am glad you have solved your issue! I noticed a few gotchas so here are a couple of things to note when importing csv files in DHIS 2:

  1. The first row

dataelement Data element not found or not accessible

is a ** funny error**, because it is actually telling you something about a single row, found here in the documentation:

"dataelement","period","orgunit","catoptcombo","attroptcombo","value","storedby","lastupd","comment"

DHIS 2 is literally complaining about the fact that it cannot find a data element with the uid “dataelement”, which is what is assumes you are trying to load in that top row. You can try yourself by loading a file with only this top line, the error will be the same, 1 ignored row, data element not found or not accessible. It is safe to ignore this error, I’d be worried if there was a UID called dataelement, which is actually a valid UID! Just extremely unlikely to pop up from the random UID generator.

  1. Ignored

Unlike the metadata importer, the data importer will report that it ignored every row on a dry run. It can make testing difficult because you don’t know if it will actually import correctly or not as it reports back that every row was ignored. Two rules of thumb here:

  1. The actual errored out row will be listed in the “Conflicts” section, even in a dry run
  2. Sadly, some errors you can only find on a real run, so always upload your files with a comment, so you can find them later via the DB to delete them if something goes wrong and all else fails
  1. File Type

UTF-8 works well and I use it for all my CSV uploads. Be wary of EXCEL’s CSV saving habits though, especially on a Mac. Your file should be UTF-8 without a BOM (a byte order marker, DHIS 2 doesn’t really care, as it will error out on row one anyway (see above)) AND it should be standard unix line feeds (LF). Excel for the Mac will default to saving it with circa 1990’s CRs only AND a BOM (unless you are editing a CSV, in which case it will keep the settings it found when it opened it). Easiest way to check is to grab a copy of Notepad++ (windows) or BBedit (mac).

BBedit:

and

Notepad++

and

Hope this helps!

Timothy Harding
Sr. Systems Analyst, BAO Systems

+1 202-536-1541 | tharding@baosystems.com | http://www.baosystems.com | Skype: hardingt@gmail.com | 2900 K Street, Suite 406, Washington D.C. 20007

On Wed, Jan 17, 2018 at 9:11 AM, Thomas Lavreys thomaslavreys@gmail.com wrote:

I found the issue. The file was saved as a csv utf-8, which is different to a normal csv (aparrently).

Thanks for the advice!

Kind regards,

Thomas

From: Adebusoye Anifalaje [mailto:busoye@baosystems.com]
Sent: Wednesday, January 17, 2018 12:31 PM

To: Thomas Lavreys thomaslavreys@gmail.com
Cc: dhis2-users@lists.launchpad.net
Subject: Re: [Dhis2-users] Data element not found or not accessible

Hi Thomas,

I aslo see that you are using version 2.23, it is possible that the csv column structure is different for this version. There might be someone on this forum who is currently working on this version and can provide input on that.

Busoye Anifalaje (PhD)
Director of Services (Principal), BAO Systems

UK: +44 7901-740-757 | US: +1 682-307-0986|

busoye@baosystems.com | http://www.baosystems.com |

Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007

On 17 Jan 2018, at 12:23, Thomas Lavreys thomaslavreys@gmail.com wrote:

I tried it anyway, and it doesn’t recognize it:

Import summary

Import count

Type

Count

Imported

0

Updated

0

Deleted

0

Ignored

13

Conflicts

Element

Description

dataelement

Data element not found or not accessible

bjDvmb4bfuf

Category option combo not found or not accessible

So I’m guessing I need to find the catoptioncombo ID somewhere.

Regards,

Thomas

From: Adebusoye Anifalaje [mailto:busoye@baosystems.com]
Sent: Wednesday, January 17, 2018 12:11 PM
To: Thomas Lavreys thomaslavreys@gmail.com
Cc: dhis2-users@lists.launchpad.net
Subject: Re: [Dhis2-users] Data element not found or not accessible

Hi Thomas,

Can you try specifying the catoptioncombo even if it is default? This was an issue with one of the version builds a while ago.

Let’s know if that helps.

Cheers

**
<image001.png>
**

Busoye Anifalaje (PhD)
Director of Services (Principal), BAO Systems

UK: +44 7901-740-757 | US: +1 682-307-0986|

busoye@baosystems.com | http://www.baosystems.com |

Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007

On 17 Jan 2018, at 12:02, Thomas Lavreys thomaslavreys@gmail.com wrote:

Dear,

I get the message “Data element not found or not accessible” when trying to import a csv file. I looked at some old threads, but failed to find an answer.

The csv file is attached (with headers, but it gave the same answer without headers).

Attached also the API page of the data element, the sharing of the data element (I have superuser access), and the orgunit page to confirm the ID. Also a screenshot of the import summary and my DHIS2 instance info.

I guess I might be missing something but I can’t figure out what exactly.

Thank you in advance for your help.

Kind regards,

Thomas Lavreys

Thomas Lavreys, MSc, PgC Trop Med

Sustainability Department SOCFINCO

Maisha Consulting bvba

Bisschoppenhof 9

3150 Tildonk

Belgium

**

Tel Socfin: +32-25-080050

Tel Maisha: +32-16-850242

Cell phone: +32-474-717641

Fax: +32-16-850242

<TEST.csv>_______________________________________________
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp


Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp


Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp


Isaiah Nyabuto
+254 729 572 616

Bsc. Health Records Management & Informatics, JKUAT.

Isaiah,

I’d love to!

Aaron

image

image

image

image

image

image

image

image

image

···

On Tue, Jan 30, 2018 at 12:25 AM, Isaiah Nyabuto isaiahnyabuto@gmail.com wrote:

Hi Aron,

Thanks for sharing your scripts and its good to know we have the same Idea.

I often write several functions to access dhis2 APIs, I realized most of this could be standardized into an R package - R dhis2 wrapper. This is also something I foresee and I would like to write in the near future. If possible, may I ask we work together on that?

Thanks.

Aaron C. White

919.522.5585

On Mon, Jan 29, 2018 at 9:00 PM, Aaron C White aaroncwhite@gmail.com wrote:

Isaiah,

I’ve found the best way is to convert rows of a CSV into JSON structure and then post to the appropriate endpoint using httr.

I’ve made a slew of helper functions that make it easy for me to do along with some markdown files starting to explain things. When I started working with dhis2, I was an analyst and most comfortable using dataframes, so this kind of fits that perspective. I’ve learned a lot along the way and plan to refactor things at some point. Eventually, I’d like to make it a real R package- just haven’t had the time!

https://github.com/aaroncwhite/dhis2R

Feel free to PM me for more detail/explanation.

Aaron


Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp


Isaiah Nyabuto
+254 729 572 616

Bsc. Health Records Management & Informatics, JKUAT.

Aaron C. White

919.522.5585

On Mon, Jan 29, 2018 at 12:24 PM, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Isaiah,

Yes, R is definitely capable of creating a file which you can import into DHIS2.

If you want to use CSV, be sure to have your file with the "row.names=FALSE " parameter and be sure to convert all “NA” values to with " na = ‘’ "

You can also post the file directly to DHIS2. Have a look at https://github.com/jason-p-pickering/dhis2-data-munging/blob/master/bootstrap/bootstrap.Rmd or more details of how to do this in the section “Loading the data elements”

Regards,

Jason


Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp

On Mon, Jan 29, 2018 at 6:04 PM, Timothy Harding tharding@baosystems.com wrote:

Isaiah:

org_kqmh_list_test.csv:

Has an extra column

1, 2, 3, etc… should not be in there unless the OUs are named 1, 2, 3.

Probably should be LF instead of CRLF:

but I’m not sure it makes a difference here.

Also you’ve got a lot of NA’s in there, not sure those are needed unless you are setting something to be NA, best just have , rather than ,NA,

Good luck!

Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+46764147049

Timothy Harding
Sr. Systems Analyst, BAO Systems

+1 202-536-1541 | tharding@baosystems.com | http://www.baosystems.com | Skype: hardingt@gmail.com | 2900 K Street, Suite 406, Washington D.C. 20007

On Mon, Jan 29, 2018 at 11:30 AM, Isaiah Nyabuto isaiahnyabuto@gmail.com wrote:

Hi all,

Thanks Timothy for sharing this. I am trying to import organisation units metadata in csv file with UTF-8 encoding. When I import the file with Metadata import in dhis2, the file is not imported at all and I do get the summary report.

I wrote the csv file with R function write.table.

Its funny enough that when I write the same file using Excel and save as csv file with UTF-8, it works well.

I do not understand what could be the problem here, has anyone tried to write a csv file with an R function that can successfully be imported in the dhis2?

Looking for your help. I have attached my csv files in this email.

Thanks

==========================
Isaiah Nyabuto
+254729572616
Bsc. Health Records Management and Informatics

On Jan 17, 2018 17:51, “Thomas Lavreys” thomaslavreys@gmail.com wrote:

Thanks for the in depth info! It will surely help along the way. I’ve only started with some testers, but I will probably run into some more issues when I load my real data.

Thanks again!!

Kind regards,

Thomas

From: Timothy Harding [mailto:tharding@baosystems.com]
Sent: Wednesday, January 17, 2018 3:35 PM
To: Thomas Lavreys thomaslavreys@gmail.com
Cc: Adebusoye Anifalaje busoye@baosystems.com; dhis2-users dhis2-users@lists.launchpad.net
Subject: Re: [Dhis2-users] Data element not found or not accessible

Hello Thomas!

I am glad you have solved your issue! I noticed a few gotchas so here are a couple of things to note when importing csv files in DHIS 2:

  1. The first row

dataelement Data element not found or not accessible

is a ** funny error**, because it is actually telling you something about a single row, found here in the documentation:

"dataelement","period","orgunit","catoptcombo","attroptcombo","value","storedby","lastupd","comment"

DHIS 2 is literally complaining about the fact that it cannot find a data element with the uid “dataelement”, which is what is assumes you are trying to load in that top row. You can try yourself by loading a file with only this top line, the error will be the same, 1 ignored row, data element not found or not accessible. It is safe to ignore this error, I’d be worried if there was a UID called dataelement, which is actually a valid UID! Just extremely unlikely to pop up from the random UID generator.

  1. Ignored

Unlike the metadata importer, the data importer will report that it ignored every row on a dry run. It can make testing difficult because you don’t know if it will actually import correctly or not as it reports back that every row was ignored. Two rules of thumb here:

  1. The actual errored out row will be listed in the “Conflicts” section, even in a dry run
  2. Sadly, some errors you can only find on a real run, so always upload your files with a comment, so you can find them later via the DB to delete them if something goes wrong and all else fails
  1. File Type

UTF-8 works well and I use it for all my CSV uploads. Be wary of EXCEL’s CSV saving habits though, especially on a Mac. Your file should be UTF-8 without a BOM (a byte order marker, DHIS 2 doesn’t really care, as it will error out on row one anyway (see above)) AND it should be standard unix line feeds (LF). Excel for the Mac will default to saving it with circa 1990’s CRs only AND a BOM (unless you are editing a CSV, in which case it will keep the settings it found when it opened it). Easiest way to check is to grab a copy of Notepad++ (windows) or BBedit (mac).

BBedit:

and

Notepad++

and

Hope this helps!

Timothy Harding
Sr. Systems Analyst, BAO Systems

+1 202-536-1541 | tharding@baosystems.com | http://www.baosystems.com | Skype: hardingt@gmail.com | 2900 K Street, Suite 406, Washington D.C. 20007

On Wed, Jan 17, 2018 at 9:11 AM, Thomas Lavreys thomaslavreys@gmail.com wrote:

I found the issue. The file was saved as a csv utf-8, which is different to a normal csv (aparrently).

Thanks for the advice!

Kind regards,

Thomas

From: Adebusoye Anifalaje [mailto:busoye@baosystems.com]
Sent: Wednesday, January 17, 2018 12:31 PM

To: Thomas Lavreys thomaslavreys@gmail.com
Cc: dhis2-users@lists.launchpad.net
Subject: Re: [Dhis2-users] Data element not found or not accessible

Hi Thomas,

I aslo see that you are using version 2.23, it is possible that the csv column structure is different for this version. There might be someone on this forum who is currently working on this version and can provide input on that.

Busoye Anifalaje (PhD)
Director of Services (Principal), BAO Systems

UK: +44 7901-740-757 | US: +1 682-307-0986|

busoye@baosystems.com | http://www.baosystems.com |

Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007

On 17 Jan 2018, at 12:23, Thomas Lavreys thomaslavreys@gmail.com wrote:

I tried it anyway, and it doesn’t recognize it:

Import summary

Import count

Type

Count

Imported

0

Updated

0

Deleted

0

Ignored

13

Conflicts

Element

Description

dataelement

Data element not found or not accessible

bjDvmb4bfuf

Category option combo not found or not accessible

So I’m guessing I need to find the catoptioncombo ID somewhere.

Regards,

Thomas

From: Adebusoye Anifalaje [mailto:busoye@baosystems.com]
Sent: Wednesday, January 17, 2018 12:11 PM
To: Thomas Lavreys thomaslavreys@gmail.com
Cc: dhis2-users@lists.launchpad.net
Subject: Re: [Dhis2-users] Data element not found or not accessible

Hi Thomas,

Can you try specifying the catoptioncombo even if it is default? This was an issue with one of the version builds a while ago.

Let’s know if that helps.

Cheers

**
<image001.png>
**

Busoye Anifalaje (PhD)
Director of Services (Principal), BAO Systems

UK: +44 7901-740-757 | US: +1 682-307-0986|

busoye@baosystems.com | http://www.baosystems.com |

Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007

On 17 Jan 2018, at 12:02, Thomas Lavreys thomaslavreys@gmail.com wrote:

Dear,

I get the message “Data element not found or not accessible” when trying to import a csv file. I looked at some old threads, but failed to find an answer.

The csv file is attached (with headers, but it gave the same answer without headers).

Attached also the API page of the data element, the sharing of the data element (I have superuser access), and the orgunit page to confirm the ID. Also a screenshot of the import summary and my DHIS2 instance info.

I guess I might be missing something but I can’t figure out what exactly.

Thank you in advance for your help.

Kind regards,

Thomas Lavreys

Thomas Lavreys, MSc, PgC Trop Med

Sustainability Department SOCFINCO

Maisha Consulting bvba

Bisschoppenhof 9

3150 Tildonk

Belgium

**

Tel Socfin: +32-25-080050

Tel Maisha: +32-16-850242

Cell phone: +32-474-717641

Fax: +32-16-850242

<TEST.csv>_______________________________________________
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp


Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp


Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp

Hi Aaron,

Thats perfect. Lets keep in touch for a skype session soon. my skype id is; Isaiah Nyabuto

Thanks.

image

image

image

image

image

image

image

image

image

···

On Tue, Jan 30, 2018 at 11:41 PM, Aaron C White aaroncwhite@gmail.com wrote:

Isaiah,

I’d love to!

Aaron

Aaron C. White

919.522.5585

On Tue, Jan 30, 2018 at 12:25 AM, Isaiah Nyabuto isaiahnyabuto@gmail.com wrote:

Hi Aron,

Thanks for sharing your scripts and its good to know we have the same Idea.

I often write several functions to access dhis2 APIs, I realized most of this could be standardized into an R package - R dhis2 wrapper. This is also something I foresee and I would like to write in the near future. If possible, may I ask we work together on that?

Thanks.

On Mon, Jan 29, 2018 at 9:00 PM, Aaron C White aaroncwhite@gmail.com wrote:

Isaiah,

I’ve found the best way is to convert rows of a CSV into JSON structure and then post to the appropriate endpoint using httr.

I’ve made a slew of helper functions that make it easy for me to do along with some markdown files starting to explain things. When I started working with dhis2, I was an analyst and most comfortable using dataframes, so this kind of fits that perspective. I’ve learned a lot along the way and plan to refactor things at some point. Eventually, I’d like to make it a real R package- just haven’t had the time!

https://github.com/aaroncwhite/dhis2R

Feel free to PM me for more detail/explanation.

Aaron


Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp


Isaiah Nyabuto
+254 729 572 616

Bsc. Health Records Management & Informatics, JKUAT.

Aaron C. White

919.522.5585

On Mon, Jan 29, 2018 at 12:24 PM, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Isaiah,

Yes, R is definitely capable of creating a file which you can import into DHIS2.

If you want to use CSV, be sure to have your file with the "row.names=FALSE " parameter and be sure to convert all “NA” values to with " na = ‘’ "

You can also post the file directly to DHIS2. Have a look at https://github.com/jason-p-pickering/dhis2-data-munging/blob/master/bootstrap/bootstrap.Rmd or more details of how to do this in the section “Loading the data elements”

Regards,

Jason


Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp

On Mon, Jan 29, 2018 at 6:04 PM, Timothy Harding tharding@baosystems.com wrote:

Isaiah:

org_kqmh_list_test.csv:

Has an extra column

1, 2, 3, etc… should not be in there unless the OUs are named 1, 2, 3.

Probably should be LF instead of CRLF:

but I’m not sure it makes a difference here.

Also you’ve got a lot of NA’s in there, not sure those are needed unless you are setting something to be NA, best just have , rather than ,NA,

Good luck!

Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+46764147049

Timothy Harding
Sr. Systems Analyst, BAO Systems

+1 202-536-1541 | tharding@baosystems.com | http://www.baosystems.com | Skype: hardingt@gmail.com | 2900 K Street, Suite 406, Washington D.C. 20007

On Mon, Jan 29, 2018 at 11:30 AM, Isaiah Nyabuto isaiahnyabuto@gmail.com wrote:

Hi all,

Thanks Timothy for sharing this. I am trying to import organisation units metadata in csv file with UTF-8 encoding. When I import the file with Metadata import in dhis2, the file is not imported at all and I do get the summary report.

I wrote the csv file with R function write.table.

Its funny enough that when I write the same file using Excel and save as csv file with UTF-8, it works well.

I do not understand what could be the problem here, has anyone tried to write a csv file with an R function that can successfully be imported in the dhis2?

Looking for your help. I have attached my csv files in this email.

Thanks

==========================
Isaiah Nyabuto
+254729572616
Bsc. Health Records Management and Informatics

On Jan 17, 2018 17:51, “Thomas Lavreys” thomaslavreys@gmail.com wrote:

Thanks for the in depth info! It will surely help along the way. I’ve only started with some testers, but I will probably run into some more issues when I load my real data.

Thanks again!!

Kind regards,

Thomas

From: Timothy Harding [mailto:tharding@baosystems.com]
Sent: Wednesday, January 17, 2018 3:35 PM
To: Thomas Lavreys thomaslavreys@gmail.com
Cc: Adebusoye Anifalaje busoye@baosystems.com; dhis2-users dhis2-users@lists.launchpad.net
Subject: Re: [Dhis2-users] Data element not found or not accessible

Hello Thomas!

I am glad you have solved your issue! I noticed a few gotchas so here are a couple of things to note when importing csv files in DHIS 2:

  1. The first row

dataelement Data element not found or not accessible

is a ** funny error**, because it is actually telling you something about a single row, found here in the documentation:

"dataelement","period","orgunit","catoptcombo","attroptcombo","value","storedby","lastupd","comment"

DHIS 2 is literally complaining about the fact that it cannot find a data element with the uid “dataelement”, which is what is assumes you are trying to load in that top row. You can try yourself by loading a file with only this top line, the error will be the same, 1 ignored row, data element not found or not accessible. It is safe to ignore this error, I’d be worried if there was a UID called dataelement, which is actually a valid UID! Just extremely unlikely to pop up from the random UID generator.

  1. Ignored

Unlike the metadata importer, the data importer will report that it ignored every row on a dry run. It can make testing difficult because you don’t know if it will actually import correctly or not as it reports back that every row was ignored. Two rules of thumb here:

  1. The actual errored out row will be listed in the “Conflicts” section, even in a dry run
  2. Sadly, some errors you can only find on a real run, so always upload your files with a comment, so you can find them later via the DB to delete them if something goes wrong and all else fails
  1. File Type

UTF-8 works well and I use it for all my CSV uploads. Be wary of EXCEL’s CSV saving habits though, especially on a Mac. Your file should be UTF-8 without a BOM (a byte order marker, DHIS 2 doesn’t really care, as it will error out on row one anyway (see above)) AND it should be standard unix line feeds (LF). Excel for the Mac will default to saving it with circa 1990’s CRs only AND a BOM (unless you are editing a CSV, in which case it will keep the settings it found when it opened it). Easiest way to check is to grab a copy of Notepad++ (windows) or BBedit (mac).

BBedit:

and

Notepad++

and

Hope this helps!

Timothy Harding
Sr. Systems Analyst, BAO Systems

+1 202-536-1541 | tharding@baosystems.com | http://www.baosystems.com | Skype: hardingt@gmail.com | 2900 K Street, Suite 406, Washington D.C. 20007

On Wed, Jan 17, 2018 at 9:11 AM, Thomas Lavreys thomaslavreys@gmail.com wrote:

I found the issue. The file was saved as a csv utf-8, which is different to a normal csv (aparrently).

Thanks for the advice!

Kind regards,

Thomas

From: Adebusoye Anifalaje [mailto:busoye@baosystems.com]
Sent: Wednesday, January 17, 2018 12:31 PM

To: Thomas Lavreys thomaslavreys@gmail.com
Cc: dhis2-users@lists.launchpad.net
Subject: Re: [Dhis2-users] Data element not found or not accessible

Hi Thomas,

I aslo see that you are using version 2.23, it is possible that the csv column structure is different for this version. There might be someone on this forum who is currently working on this version and can provide input on that.

Busoye Anifalaje (PhD)
Director of Services (Principal), BAO Systems

UK: +44 7901-740-757 | US: +1 682-307-0986|

busoye@baosystems.com | http://www.baosystems.com |

Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007

On 17 Jan 2018, at 12:23, Thomas Lavreys thomaslavreys@gmail.com wrote:

I tried it anyway, and it doesn’t recognize it:

Import summary

Import count

Type

Count

Imported

0

Updated

0

Deleted

0

Ignored

13

Conflicts

Element

Description

dataelement

Data element not found or not accessible

bjDvmb4bfuf

Category option combo not found or not accessible

So I’m guessing I need to find the catoptioncombo ID somewhere.

Regards,

Thomas

From: Adebusoye Anifalaje [mailto:busoye@baosystems.com]
Sent: Wednesday, January 17, 2018 12:11 PM
To: Thomas Lavreys thomaslavreys@gmail.com
Cc: dhis2-users@lists.launchpad.net
Subject: Re: [Dhis2-users] Data element not found or not accessible

Hi Thomas,

Can you try specifying the catoptioncombo even if it is default? This was an issue with one of the version builds a while ago.

Let’s know if that helps.

Cheers

**
<image001.png>
**

Busoye Anifalaje (PhD)
Director of Services (Principal), BAO Systems

UK: +44 7901-740-757 | US: +1 682-307-0986|

busoye@baosystems.com | http://www.baosystems.com |

Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007

On 17 Jan 2018, at 12:02, Thomas Lavreys thomaslavreys@gmail.com wrote:

Dear,

I get the message “Data element not found or not accessible” when trying to import a csv file. I looked at some old threads, but failed to find an answer.

The csv file is attached (with headers, but it gave the same answer without headers).

Attached also the API page of the data element, the sharing of the data element (I have superuser access), and the orgunit page to confirm the ID. Also a screenshot of the import summary and my DHIS2 instance info.

I guess I might be missing something but I can’t figure out what exactly.

Thank you in advance for your help.

Kind regards,

Thomas Lavreys

Thomas Lavreys, MSc, PgC Trop Med

Sustainability Department SOCFINCO

Maisha Consulting bvba

Bisschoppenhof 9

3150 Tildonk

Belgium

**

Tel Socfin: +32-25-080050

Tel Maisha: +32-16-850242

Cell phone: +32-474-717641

Fax: +32-16-850242

<TEST.csv>_______________________________________________
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp


Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp


Mailing list: https://launchpad.net/~dhis2-users

Post to : dhis2-users@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-users

More help : https://help.launchpad.net/ListHelp


Isaiah Nyabuto
+254 729 572 616

Bsc. Health Records Management & Informatics, JKUAT.