Hello everyone,
We are currently running DHIS2 2.39.1.2
We created a survey in DHIS2 using an event program. The users used the Capture app to collect the data. Most of the fields are drop-down menus created using option sets. We are done with data collection and now cleaning.
On Monday 24th July, we noticed that there appeared to be some crosslinking/mismatch of the data in the columns using both the line listing app and the event reports app
Last weekend we tried to upgrade to the latest version of 2.39 but it’s been having errors. A rollback was done using a backed-up version of the database.
Could anybody please provide us some insight into what could be going on and how to fix it?
Thanks for reporting this. We need to understand several things here. First, if the information is stored correctly but only during the display of these tables the mismatch starts to occur or not? To check for this, it might be easier to check the API, could you please open the Network tab (F12 → Network) and then display the table. See the network request that’s being sent to retrieve the info for the table, it will probably be in JSON, do you see the mismatch in the API as well?
If you don’t see a mismatch in the API request then this might be a bug in the app, but if you see a mismatch then we need to check again during the data entry.
Second, to check that the issue is not during the data entry, go to the capture app and enter the data, but before submitting the data please open the network tab again then submit the data. Check for the request where the data is being sent. Do you see a mismatch there?
If there is no mismatch during the data entry and when checking the API then it might be from the app.
What version of the app are you using? Additionally, to be sure if there isn’t something else, during the generation of analytics, do you see any errors in the Catalina.out log?
I hope these tips help us figure out what’s the real issue and find a solution. Thank you!
Please what version of the Data Capture app are you using? Is the mismatch in the Data Visuzalizer?
@Gassim Yes I did. However, I needed the expertise of some of my colleagues to perform some of the tests and they are currently not available.
Here is the interesting thing: we upgraded a test instance to 2.40 and we restored the database to it.
I tried to pull the survey data from the Line listing app and got the same mismatched data. However, I am able to pull the correctly formatted data using the event reports app. There are a few challenges with using the event reports app though:
I can’t pull all the data elements (70+) at once. I have to pull data elements in batches. Even with that, I cannot pull more than a certain number of data elements too.
The version 2.40 has changed the API request string structure. Before we could just form required fields list like &dimension.program_uid.DE_uid. Now if we need to get all data ordered in certain way, we have to enlist all that DEs and dhis2 fields(ouname, eventdate etc) after the &headers= section
It has good and bad parts.
Good one:
a) in &headers= section you can enlist only the DHIS2 and other DEs/TEAs fields what you need
b) it is no need anymore to repeat &dimension for each DE/TEA
c) if you don’t need a specific order of the DE/TEA in the request result, the request header can be shorter
Bad one: you have to enlist all the DEs\TEAs after &headers= section and that’s make your request string 2 time longer if you need a specific order. It affects TOMCAT configuration value for maxHttpHeaderSize, which should be set and re-run again
Currently the developers documentation has not been updated yet in more extended way on how to use new API request.
As an example: I have an API request string for 173 DEs\TEAs and it has a length 6102 bytes. When I created the same request string by using new API rules I got the length 8047 bytes. And as a result I had to split that request string or just change the server settings. Therefore most of time when you migrate from older version of DHIS2 to version 2.40 and trying to use Line Listing app, some reports, which has more than 150 DEs will fail or will not produce any results.
In these situation I want to propose to the DHIS2 developers few points on how to form a request string, which I believe would be more convenient for everyone:
a) &headers= section should contain only the dhis2 fields. This can help to reduce enlisting an extra fields. All fields enlisted here, should be placed before the rest requested DEs/TEAs.
b) &dimension= section should be taken as a list of elements to be returned and the order should be preserved as it was enlisted in the request string.
c) if we already requesting events in the beginning of the request as/api/analytics/events/query/ maybe we should just omit\delete &outputType=EVENT?
d) what is the point of having &stage=stage_uid if we add all those stage_uid before each DEs?
Hope I could provide some explanations and these points could be useful
