Best practices, Import of external data

Hello All!
I am very very new at DHIS2 and want to import data from external systems - starting with datasets generated elsewhere, starting simply with aggregated data capture. It means I have to create a category/data element structure that more or less matches the data I get.

I would really like a references to best practices for bulk importing data - I am not properly proficient at searching all the resources available.

I have found an Data Import Wizard from HISP Uganda - but I cannot download documentation from there.

I have also tried to install version 1.0.9 from the the App Hub - is that the way I should go? It indeed seems promising - but I wanted to check if I am on the best path!

One of the reasons for my questions is that I found two githubs repos for it, the newest is for the data-import-wizard v2 but the app hub gave me 1.09

If nothing else, maybe this question can help other acolytes later!

2 Likes

Hi @gutorm

Good question… Import / Export questions are one of the most common questions asked in the CoP. You mentioned apps, so in the DHIS2 App Hub there are two apps (I don’t have a preference):

For some reason, I’ve found myself always referring to and using the Import / Export app. Import/Export - DHIS2 Documentation

In the Import / Export app, one would start with exporting and importing metadata (OUs, data elements, category options, data sets…etc) and then exporting the metadata dependency (selecting the data set and then importing it). One tip I learned is that anything that’s required during the creation of the metadata when using the UI will normally be required during the import of data so when not sure the required fields are all in the UI. The CSV templates in the developer api documentation is helpful to check for the required fields as well.

I hope this helps! Please feel free to discuss this further… thanks!

1 Like

Thanks @Gassim
The app-hub from the DHIS2-server is slightly different - it only has one.

image

It would be very kind if you or anyone else could answer, as I am still not sure what do use:

The BAO Import App you showed a picture seems to be only for metadata, I have those in place and would import the (aggragated) data corresponding to a data-set I have already made.

The Import/Export - DHIS2 Documentation you referred to seems to be used from DHIS2 to DHIS2 - or am I missing something?

I will play some more around with the Uganda-app, but I wanted to make sure I using the ‘right’ tool - is it the one and only then?

2 Likes

You are right. The other app needs to be manually uploaded from apps.dhis2.org The same for another app that I forgot to mention, Bulk Load app. True, the BIF app from BAO seems to be dedicated for metadata only.

Actually, this Import / Export app uploads the files you select such as CSV or JSON regardless of the source but as long as the template is correct.

I think the Data Import Wizard Uganda-app is user friendly and as long as the imports are successful then it’s the right tool to use but it’s not the only option. Import / Export app is the official app from the software core team for both metadata and aggregate data.

It might be a good idea to test in a test instance first and check the Catalina.out log for any errors after the import.

2 Likes

By ‘template’ - do you mean that the import file needs a specific format? Like sporting ID’s that match the entries in the datamodell?
So I basically will have to (by some script/app) transform the data so it will work in a setting where the csv-files contain 3 attributes? (As opposed to matching som data-set)

So sorry for being slow here, it seems that there is something I miss completely. (The Uganda-app has some ‘mapping’ parameters from one column to another)

image

1 Like