CSV import worked magically for me when importing 13 000 rows of population data.
However, I’m completely perplexed about the behavior of the importer on my other data. Here’s an example of what I tried.
Import 35 rows of values for one variable -> fail (‘no data values found’)
Import 2 rows of values for one variable -> success
Import those 2 rows again plus 10 more -> success
Repeat process until all 35 original rows are in -> success
Re-import original 35 rows -> success
So it’s almost as if the importer can only import a small number at a time, but can update many? This certainly doesn’t explain my population dataset, but I can’t really figure out this erratic behavior. Also the report when it does work is odd giving sometimes fewer numbers next to “imported” than I actually did import (these are then added to the “updated” count so the total always seems to be accurate.)
Is anyone having similar trouble?
Tom
···
This message was scanned for viruses with Trend Micro ScanMail, GFI MailSecurity and GFI MailEssentials by the World Health Organization Regional Office for the Western Pacific. However, the recipient is advised to scan this e-mail and any attached files for viruses.
Disclaimer:
This e-mail, together with any attachments, is intended for the named recipients only and is confidential. It may also be privileged or otherwise protected by law.
If you have received it in error, please notify the sender immediately by reply e-mail and delete it and any attachments from your system. You may not copy or disclose its contents to anyone.
I have had problems importing in earlier versions of DHIS but I am
about at the point with my current project where I need this to "just
work".
I have exported CSV records from an instance and then been unable to
re-import the same records.
I do hope this gets sorted out.
I'll be working on setting up a new v2.8 instance and trying to import
again and I'll report my progress (or lack thereof).
My greatest frustration has been the lack of comprehensible error
messages and intelligent error handling. When it fails, it would be
good to get a nice log of the problem (the record where it failed, the
cause of the failure) and it would be nice if it would attempt to
continue with the rest of the records. Past error logs have been very
generic and difficult to sort out.
.Mark
···
--
Mark Spohr, MD
mhspohr@gmail.com
+1 530 554 2230
On Tue, May 8, 2012 at 7:58 PM, Hiatt, Mr Tom (WPRO) <hiattt@wpro.who.int> wrote:
CSV import worked magically for me when importing 13 000 rows of population
data.
However, I’m completely perplexed about the behavior of the importer on my
other data. Here’s an example of what I tried.
- Import 35 rows of values for one variable -> fail (‘no data
values found’)
- Import 2 rows of values for one variable -> success
- Import those 2 rows again plus 10 more -> success
- Repeat process until all 35 original rows are in -> success
- Re-import original 35 rows -> success
So it’s almost as if the importer can only import a small number at a time,
but can update many? This certainly doesn’t explain my population dataset,
but I can’t really figure out this erratic behavior. Also the report when it
does work is odd giving sometimes fewer numbers next to “imported” than I
actually did import (these are then added to the “updated” count so the
total always seems to be accurate.)
Is anyone having similar trouble?
Tom
________________________________
This message was scanned for viruses with Trend Micro ScanMail, GFI
MailSecurity and GFI MailEssentials by the World Health Organization
Regional Office for the Western Pacific. However, the recipient is advised
to scan this e-mail and any attached files for viruses.
Disclaimer:
This e-mail, together with any attachments, is intended for the named
recipients only and is confidential. It may also be privileged or otherwise
protected by law.
If you have received it in error, please notify the sender immediately by
reply e-mail and delete it and any attachments from your system. You may not
copy or disclose its contents to anyone.
Remember that the data value import relies on the meta-data identifiers / codes used in the exchange file are already present in the destination database.
Are you setting any import options?
I am available on google chat on this email and on skype as larshelgeoverland . Please contact me so we can try to sort this out.
Just a quick update. I now tried importing from csv for another 2 past years. The import seems to have gone through without any problems. The only odd thing is in the import summary it says 78 of the 475 records were “updated” rather than “imported” which as far as I know can’t be correct, because I haven’t stored any data for those years yet in the database. But anyway, they’re in an that’s the important part!
Remember that the data value import relies on the meta-data identifiers / codes used in the exchange file are already present in the destination database.
Are you setting any import options?
I am available on google chat on this email and on skype as larshelgeoverland . Please contact me so we can try to sort this out.
regards, Lars
This message was scanned for viruses with Trend Micro ScanMail, GFI MailSecurity and GFI MailEssentials by the World Health Organization Regional Office for the Western Pacific. However, the recipient is advised to scan this e-mail and any attached files for viruses.
Disclaimer:
This e-mail, together with any attachments, is intended for the named recipients only and is confidential. It may also be privileged or otherwise protected by law.
If you have received it in error, please notify the sender immediately by reply e-mail and delete it and any attachments from your system. You may not copy or disclose its contents to anyone.
A data value is considered an update if there exists another data value in the system with the same composite primary id (data element, period, org unit, category option combo). If you can reproduce the problem and want to investigate this further we can chat on gmail/skype.
Just a quick update. I now tried importing from csv for another 2 past years. The import seems to have gone through without any problems. The only odd thing is in the import summary it says 78 of the 475 records were “updated” rather than “imported” which as far as I know can’t be correct, because I haven’t stored any data for those years yet in the database. But anyway, they’re in an that’s the important part!
Thanks Lars. I haven’t been able to reproduce it since, as you note, once a record has been made, it’s an update so I would have to go in somehow and delete records, which I don’t know how to do. Anyhow, for my purposes, it’s working fine, so I don’t think we need to investigate further.
Just a quick update. I now tried importing from csv for another 2 past years. The import seems to have gone through without any problems. The only odd thing is in the import summary it says 78 of the 475 records were “updated” rather than “imported” which as far as I know can’t be correct, because I haven’t stored any data for those years yet in the database. But anyway, they’re in an that’s the important part!
Thanks,
Tom
That is good.
A data value is considered an update if there exists another data value in the system with the same composite primary id (data element, period, org unit, category option combo). If you can reproduce the problem and want to investigate this further we can chat on gmail/skype.
Lars
This message was scanned for viruses with Trend Micro ScanMail, GFI MailSecurity and GFI MailEssentials by the World Health Organization Regional Office for the Western Pacific. However, the recipient is advised to scan this e-mail and any attached files for viruses.
Disclaimer:
This e-mail, together with any attachments, is intended for the named recipients only and is confidential. It may also be privileged or otherwise protected by law.
If you have received it in error, please notify the sender immediately by reply e-mail and delete it and any attachments from your system. You may not copy or disclose its contents to anyone.