Importing event data - empty "Due date" column

Good day all,

We use a custom-developed handheld device to collect data at the community-level, which is then downloaded in a DHIS2-appropriate CSV format using an external program. The CSV file is subsequently imported into the DHIS2 instance using event import.

The external program creates a Tracker capture CSV file that populates the “Event date” column, but leaves the “Due date” column empty. This format has been using for at least 4 years without any problems.

We have recently upgraded our instance to 2.33.10, after which our event import failed and simply gave the message “Error”. Through a process of elimination we realised that the CSV requires the “Due date” column to be populated for the import to succeed after our most recent instance upgrade.

As it will be nearly impossible for us to update the external program that generates the CSV’s, is it possible to have the upgraded instance once again “disregard” the “Due date” column in the CSV?

Any guidance/advice/assistance would be greatly appreciated.

Thank you,

Andre

Thank you for your post @ACoetzer1988! This must be a challenge indeed; however, I hope we’ll be able to resolve it. Are you sure the cause is the due date because when I check the api schema it says that dueDate is not required, see (https://play.dhis2.org/2.37.2/api/schemas/programStageInstance?fields=displayName,properties[fieldName,required])

The event date is required true, and then the due date will be generated either based on incident date or enrollment date; however, I wonder if creating a program rule to calculate a “due date” (even if a random one for now) would work during the import? It probably should.

Finally, I’ve not given you a specific answer because it’s supposed to be ‘disregarded’ already; however, if this is reproducible in any of the latest instances that you can see in play.dhis2.org then it’s probably a bug that needs fixing (: We can create a jira.dhis2.org issue for it. :+1:

Hi @Gassim

Thanks for the quick reply and information.

I just double checked and it is most definitely the empty “Due Date” column in the CSV that prevents the import from being successful. I agree that it is really strange as it should not be happening, and would like to try the program rule as a temporary solution while I look at latest instances as you have suggested.

Could you perhaps give me some guidance as to how the program rule should be set up? More specifically - what should the Program rule actions be?

Thank you once again!

Andre

Hi @Gassim

I tried exporting and immediately re-importing event data using DHIS2 play instances as suggested. On 2.34.9 I also got “Error” but testing it on 2.36.6 and 2.37.2 gave the “error” message and also more information - “Invalid event due date” (see screenshots).


So it is clear that the Event due date has been made a requirement for the event import to succeed. Or there is a bug that is at play since the system does not export the event data in a format that can be imported (exports without any content in the “due date” column).

Any thoughts on this?

Kind regards,

Andre

Thank you for trying this in play! Actually, there is a similar case with orgunits, the exported files is different from the required import parameters:

Similarly, I think you’d have to use the following CSV columns in the docs: Tracker - DHIS2 Documentation

I hope this will work!

Hi @Gassim,

Thanks for the suggestion.

I just double checked the documentation and the heading orders in the export and import file and both are as suggested by the documentation (screenshots below).

Plus, that would not explain why testing it on the play instances gave the specific error of “Invalid event due date”, which is corrected once that column is populated.

Required order

Export/Import format

Do you have any guidance for the program rule as mentioned in an earlier post so that I can test that?

Thank you once again for the guidance, I appreciate it!

Andre

1 Like

Thank you for your patience! You are correct. This is a bug…

So I’ll quote @Gintare from @tracker-backend:

is it possible that the date for some events in the list is provided, but doesn’t comply with the format? With csv, empty cells are probably not considered null, and then date validation fails. I would say it’s a bug

Would you like to create a Jira bug issue for this (or I can create it on your behalf if you want) jira.dhis2.org?

As for the program rule, I’m not entirely sure but I’d have used this rule (change 3 to any number):
d2:addDays(V{due_date},‘3’)

[I’m not sure PR work during the import-.- so I’m asking a question on the CoP as well (Program Rules work during/after import?)]

Hi @Gassim,

Thanks for the feedback and for confirming the bug. I will create a JIRA issue with a link to the COP as well.

For the program rule itself, thank you for confirming that it will work in the other COP that you linked. Thats great news as I can at least focus on that while the JIRA issue is (hopefully) resolved.

Just one question - would the program rule need a "Program rule actions? If so, how do I set that up?

Thanks!

Andre

Hi there,

Regarding Incident date - if you don’t use it during the tracker data collection, it will take current date and time when the record created. I had a real challenge with this as the DHIS instance used in many countries with different time zones.
It affects your tracker if you disable use of future dates in your tracker, as the local time may have time difference with the server time.
So take this one into account as well.
Regards, Ulanbek

1 Like

Very important info! Thank you for sharing! :+1:

Hello All,
I’m facing the same problem. Could you please help me with any feedback?