CSV Event upload failed: "latitude" not a valid Double value

Jesus_Solano-Roman · 4 August 2017 19:27

Hello all.

I’ve constructed a csv file to bulk import Events to a local instance of DHIS2 as per guidelines from the documentation . My curl command to upload is the following:

curl --data-binary @programKenemaTest.csv “http://localhost:8085/api/events” -H “Content-Type:application/csv” -u admin:district -v

however, when uploading from terminal I get the following error message:

{“httpStatus”:“Internal Server Error”,

“httpStatusCode”:500,“status”:“ERROR”,

“message”:"Can not construct instance of java.lang.Double from String value ("latitude"): not a valid Double value\n

at [Source: com.fasterxml.jackson.dataformat.csv.impl.UTF8Reader@47005012; line: 1, column: 88] (through reference chain: org.hisp.dhis.dxf2.events.event.csv.CsvEventDataValue["latitude"])"}

(Rest of response from server can be seen here: https://codeshare.io/5QmLvJ)

The value for latitude (and longitude) in this particular dataset is 0. I generated this dataset and the CSV file using R. The type of value for those two columns evaluates to “double”, and I have exported the CSV using UTF-8 encoding:

write.csv(data, file = “data.csv”, row.names = F, fileEncoding = ‘UTF-8’)

Any ideas why this file cannot be parsed correctly? Entries created inside DHIS2 using the Event Capture app have latitude and longitude values of 0 without a problem. I’ve attached the csv file if it’s of any help.

Thanks for your time,

Antonio Solano.

programKenemaTest.csv (980 Bytes)

Jesus_Solano-Roman · 4 August 2017 19:41

UPDATE: I noticed that the csv file I downloaded from DHIS2 with a sample of the Event data does not contain quotations so I did the same on the exported file from R

write.csv(data, file = “data.csv”, row.names = F, fileEncoding = ‘UTF-8’, quote = F)

But the upload still fails. Also If I upload manually using the import function inside DHIS2, the dry run shows the following conflicts:

event
Data element not found or not accessible
tHF7qDZVeoT
Data element not found or not accessible

and all elements are ignored.

Thanks again.

···

On Fri, Aug 4, 2017 at 3:27 PM, Jesus Solano-Roman asolano@broadinstitute.org wrote:

Hello all.

I’ve constructed a csv file to bulk import Events to a local instance of DHIS2 as per guidelines from the documentation . My curl command to upload is the following:

curl --data-binary @programKenemaTest.csv “http://localhost:8085/api/events” -H “Content-Type:application/csv” -u admin:district -v

however, when uploading from terminal I get the following error message:

{“httpStatus”:“Internal Server Error”,

“httpStatusCode”:500,“status”:“ERROR”,

“message”:"Can not construct instance of java.lang.Double from String value ("latitude"): not a valid Double value\n

at [Source: com.fasterxml.jackson.dataformat.csv.impl.UTF8Reader@47005012; line: 1, column: 88] (through reference chain: org.hisp.dhis.dxf2.events.event.csv.CsvEventDataValue["latitude"])"}

(Rest of response from server can be seen here: https://codeshare.io/5QmLvJ)

The value for latitude (and longitude) in this particular dataset is 0. I generated this dataset and the CSV file using R. The type of value for those two columns evaluates to “double”, and I have exported the CSV using UTF-8 encoding:

write.csv(data, file = “data.csv”, row.names = F, fileEncoding = ‘UTF-8’)

Any ideas why this file cannot be parsed correctly? Entries created inside DHIS2 using the Event Capture app have latitude and longitude values of 0 without a problem. I’ve attached the csv file if it’s of any help.

Thanks for your time,

Antonio Solano.

Jesus_Solano-Roman · 4 August 2017 20:06

SECOND UPDATE: Sorry to keep updating this but I cannot edit the original post.

I realized that the header had to be removed and also that the last two columns should be left empty (storedBy, ProvidedElsewhere).

I believe this has fixed the issue. However, one last question:

Upload was successful for 8/9 entries the DE F3ogKBuviRA is a coordinate data element, and I’ve tried using two formats: “[-11.94,8.84]” and “-11.94,8.84”. (not using quotation marks). These both fail on import. What is wrong with the format of my coordinates? The Documentation says:

Coordinate: A point coordinate specified as longitude/latitude in decimal degrees. All coordinate should be specified in the format “-19.23 , 56.42” with a comma separating the longitude and latitude. (Since v2.21).

My system is

Version:

2.27

Build revision:

b531614

Thanks again and apologies for the constant updates.

···

On Fri, Aug 4, 2017 at 3:41 PM, Jesus Solano-Roman asolano@broadinstitute.org wrote:

UPDATE: I noticed that the csv file I downloaded from DHIS2 with a sample of the Event data does not contain quotations so I did the same on the exported file from R

write.csv(data, file = “data.csv”, row.names = F, fileEncoding = ‘UTF-8’, quote = F)

But the upload still fails. Also If I upload manually using the import function inside DHIS2, the dry run shows the following conflicts:

event
Data element not found or not accessible
tHF7qDZVeoT
Data element not found or not accessible

and all elements are ignored.

Thanks again.

On Fri, Aug 4, 2017 at 3:27 PM, Jesus Solano-Roman asolano@broadinstitute.org wrote:

Hello all.

I’ve constructed a csv file to bulk import Events to a local instance of DHIS2 as per guidelines from the documentation . My curl command to upload is the following:

curl --data-binary @programKenemaTest.csv “http://localhost:8085/api/events” -H “Content-Type:application/csv” -u admin:district -v

however, when uploading from terminal I get the following error message:

{“httpStatus”:“Internal Server Error”,

“httpStatusCode”:500,“status”:“ERROR”,

“message”:"Can not construct instance of java.lang.Double from String value ("latitude"): not a valid Double value\n

at [Source: com.fasterxml.jackson.dataformat.csv.impl.UTF8Reader@47005012; line: 1, column: 88] (through reference chain: org.hisp.dhis.dxf2.events.event.csv.CsvEventDataValue["latitude"])"}

(Rest of response from server can be seen here: https://codeshare.io/5QmLvJ)

The value for latitude (and longitude) in this particular dataset is 0. I generated this dataset and the CSV file using R. The type of value for those two columns evaluates to “double”, and I have exported the CSV using UTF-8 encoding:

write.csv(data, file = “data.csv”, row.names = F, fileEncoding = ‘UTF-8’)

Any ideas why this file cannot be parsed correctly? Entries created inside DHIS2 using the Event Capture app have latitude and longitude values of 0 without a problem. I’ve attached the csv file if it’s of any help.

Thanks for your time,

Antonio Solano.

morten · 6 August 2017 17:51

Hi there

I’m not sure about this issue, but there have been some fixes to CSV import lately, would you mind getting the latest 227 and retry? if it’s still present, could you write up a JIRA issue for? and we will have a look

jira.dhis2.org

···

On Fri, Aug 4, 2017 at 10:06 PM, Jesus Solano-Roman asolano@broadinstitute.org wrote:

SECOND UPDATE: Sorry to keep updating this but I cannot edit the original post.

I realized that the header had to be removed and also that the last two columns should be left empty (storedBy, ProvidedElsewhere).

I believe this has fixed the issue. However, one last question:

Upload was successful for 8/9 entries the DE F3ogKBuviRA is a coordinate data element, and I’ve tried using two formats: “[-11.94,8.84]” and “-11.94,8.84”. (not using quotation marks). These both fail on import. What is wrong with the format of my coordinates? The Documentation says:

Coordinate: A point coordinate specified as longitude/latitude in decimal degrees. All coordinate should be specified in the format “-19.23 , 56.42” with a comma separating the longitude and latitude. (Since v2.21).

My system is

Version:

2.27

Build revision:

b531614

Thanks again and apologies for the constant updates.

Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

–

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Fri, Aug 4, 2017 at 3:41 PM, Jesus Solano-Roman asolano@broadinstitute.org wrote:

UPDATE: I noticed that the csv file I downloaded from DHIS2 with a sample of the Event data does not contain quotations so I did the same on the exported file from R

write.csv(data, file = “data.csv”, row.names = F, fileEncoding = ‘UTF-8’, quote = F)

But the upload still fails. Also If I upload manually using the import function inside DHIS2, the dry run shows the following conflicts:

event
Data element not found or not accessible
tHF7qDZVeoT
Data element not found or not accessible

and all elements are ignored.

Thanks again.

On Fri, Aug 4, 2017 at 3:27 PM, Jesus Solano-Roman asolano@broadinstitute.org wrote:

Hello all.

I’ve constructed a csv file to bulk import Events to a local instance of DHIS2 as per guidelines from the documentation . My curl command to upload is the following:

curl --data-binary @programKenemaTest.csv “http://localhost:8085/api/events” -H “Content-Type:application/csv” -u admin:district -v

however, when uploading from terminal I get the following error message:

{“httpStatus”:“Internal Server Error”,

“httpStatusCode”:500,“status”:“ERROR”,

“message”:"Can not construct instance of java.lang.Double from String value ("latitude"): not a valid Double value\n

at [Source: com.fasterxml.jackson.dataformat.csv.impl.UTF8Reader@47005012; line: 1, column: 88] (through reference chain: org.hisp.dhis.dxf2.events.event.csv.CsvEventDataValue["latitude"])"}

(Rest of response from server can be seen here: https://codeshare.io/5QmLvJ)

The value for latitude (and longitude) in this particular dataset is 0. I generated this dataset and the CSV file using R. The type of value for those two columns evaluates to “double”, and I have exported the CSV using UTF-8 encoding:

write.csv(data, file = “data.csv”, row.names = F, fileEncoding = ‘UTF-8’)

Any ideas why this file cannot be parsed correctly? Entries created inside DHIS2 using the Event Capture app have latitude and longitude values of 0 without a problem. I’ve attached the csv file if it’s of any help.

Thanks for your time,

Antonio Solano.

Jesus_Solano-Roman · 7 August 2017 15:59

Hi Morten.

I have updated to the lastest version available in Docker:

Version:
2.27
Build revision:
068f643
Build date:
2017-08-05 11:20

Unfortunately this has made things worse apparently. Trying to upload -any- rows in CSV is now not working. For instance, when trying to upload this csv:

tHF7qDZVeoT,ACTIVE,iMQ9InaUU5m,Wn7LAv17lfS,kJq2mPyFEHo,2014-05-25T00:00:00.000,2017-07-12T19:11:19.801,0,0,oZg33kd9taw,Male,

Using

curl -d @programKenemaTest.csv “http://localhost:8085/api/events” -H “Content-Type:text/csv” -u admin:district -v

I get the error:

“httpStatusCode”:500,“status”:“ERROR”,“message”:“could not execute statement”

Same error when I use the import event app in DHIS2.

I converted that row into XML and the import was successful. So before I create the issue in JIRA, I would prefer if you take a look at the format of my CSV. If it seems fine to you, then there must be an error in the CSV import functions.

Thanks for the help.

Antonio Solano-Roman

Data Visualization & User Experience Design

Sabeti Lab - Broad Institute

···

On Aug 6 2017, at 1:51 pm, Morten Olav Hansen morten@dhis2.org wrote:

Hi there

I’m not sure about this issue, but there have been some fixes to CSV import lately, would you mind getting the latest 227 and retry? if it’s still present, could you write up a JIRA issue for? and we will have a look

jira.dhis2.org

–

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Fri, Aug 4, 2017 at 10:06 PM, Jesus Solano-Roman asolano@broadinstitute.org wrote:

SECOND UPDATE: Sorry to keep updating this but I cannot edit the original post.

I realized that the header had to be removed and also that the last two columns should be left empty (storedBy, ProvidedElsewhere).

I believe this has fixed the issue. However, one last question:

Upload was successful for 8/9 entries the DE F3ogKBuviRA is a coordinate data element, and I’ve tried using two formats: “[-11.94,8.84]” and “-11.94,8.84”. (not using quotation marks). These both fail on import. What is wrong with the format of my coordinates? The Documentation says:

Coordinate: A point coordinate specified as longitude/latitude in decimal degrees. All coordinate should be specified in the format “-19.23 , 56.42” with a comma separating the longitude and latitude. (Since v2.21).

My system is

Version:

2.27

Build revision:

b531614

Thanks again and apologies for the constant updates.

On Fri, Aug 4, 2017 at 3:41 PM, Jesus Solano-Roman asolano@broadinstitute.org wrote:

UPDATE: I noticed that the csv file I downloaded from DHIS2 with a sample of the Event data does not contain quotations so I did the same on the exported file from R

write.csv(data, file = “data.csv”, row.names = F, fileEncoding = ‘UTF-8’, quote = F)

But the upload still fails. Also If I upload manually using the import function inside DHIS2, the dry run shows the following conflicts:

event
Data element not found or not accessible
tHF7qDZVeoT
Data element not found or not accessible

and all elements are ignored.

Thanks again.

On Fri, Aug 4, 2017 at 3:27 PM, Jesus Solano-Roman asolano@broadinstitute.org wrote:

Hello all.

I’ve constructed a csv file to bulk import Events to a local instance of DHIS2 as per guidelines from the documentation . My curl command to upload is the following:

curl --data-binary @programKenemaTest.csv “http://localhost:8085/api/events” -H “Content-Type:application/csv” -u admin:district -v

however, when uploading from terminal I get the following error message:

{“httpStatus”:“Internal Server Error”,

“httpStatusCode”:500,“status”:“ERROR”,

“message”:"Can not construct instance of java.lang.Double from String value ("latitude"): not a valid Double value\n

at [Source: com.fasterxml.jackson.dataformat.csv.impl.UTF8Reader@47005012; line: 1, column: 88] (through reference chain: org.hisp.dhis.dxf2.events.event.csv.CsvEventDataValue["latitude"])"}

(Rest of response from server can be seen here: https://codeshare.io/5QmLvJ)

The value for latitude (and longitude) in this particular dataset is 0. I generated this dataset and the CSV file using R. The type of value for those two columns evaluates to “double”, and I have exported the CSV using UTF-8 encoding:

write.csv(data, file = “data.csv”, row.names = F, fileEncoding = ‘UTF-8’)

Any ideas why this file cannot be parsed correctly? Entries created inside DHIS2 using the Event Capture app have latitude and longitude values of 0 without a problem. I’ve attached the csv file if it’s of any help.

Thanks for your time,

Antonio Solano.

Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

tiha · 24 August 2021 12:59

Working on 2.36 and still have issues importing tracker events. Any known issues with the Import/Export app

Gassim · 25 August 2021 15:56

Hey! May I ask what issues are you facing? What is the version of the DHIS2 instance that you are using? Would you share the logs?
As you see this is a topic from 2017 - many changes have happened ever since. I’d actually suggest creating a new topic if you can’t find a similar issue to yours on the CoP.
You can also find out more details about bugs on Jira as well but you are recommended to start a topic here as to find a solution to the issue.