Limitations of Translation API

Development - Développement
1

Hello DHIS2 developers,

I work for TerraFrame, an organisation which is developing a DHIS2 integration which synchronizes entire hierarchies of Organisation Units over to DHIS2. I have run into a few problems which are due to the specifics of how the DHIS2 translation and metadata API works. These problems are slowing down our synchronization process to a max synchronization speed of about 400 organisation units per hour.

There are essentially three problems I have run into which prevent me from synchronizing data in a performant way.

  1. I am unable to submit translation data for Organisation Units using the metadata API. Technically this is supported, however when I attempt to do so while updating an existing object, if a translation with the same Property and Locale pair already exists, instead of updating it, the DHIS2 instead creates a duplicate value, effectively corrupting the database. For an update such as this, I would expect the DHIS2 to simply update the existing value with the value which I am providing. The documentation explicitly mentions that the metadata import endpoint is not supposed to allow for updating of translations, “as it would be too easy to make mistakes and overwrite the other available locales.”. As a result, I must submit my data again to a separate endpoint, the entity translations endpoint, which causes me to perform another request, which significantly slows down my synchronization.

  2. This separate endpoint requires that I have knowledge of existing data within your database: the translations array for a given Organisation Unit. Our system does not have this knowledge, and since this data must be resubmitted it requires us to fetch this information and then make the modifications we need to it, and then submit it to the entity translations endpoint. Ideally we would be able to simply submit the data which we have, and DHIS2 would perform the necessary merging into the existing data.

  3. When submitting an update to an existing Organisation Unit via the metadata API, I am required to also submit: name, shortName, openingDate, when really only id should be required. Again, this suffers from the same problem mentioned in #2: our system may not know that information, which requires us to fetch the object or else risk accidentally changing it.

Thanks

2 Likes
2

@Richard_Rowlands thanks for reaching out! Can you share the specific API calls you are making where you see these issues?

1 Like
3

Hi Austin,

Thanks for the quick reply. The following curl request can be used to reproduce issue #1 on a DHIS2 play server with default database:

curl -X POST -d '{"organisationUnits":[{"name":"Sierra Leone","shortName":"Sierra Leone","openingDate":"1994-01-01T00:00:00.000","id":"ImspTQPwCqd","translations":[{"property":"NAME","value":"test merge mode","locale":"en_GB"}]}]}' -H "Content-Type: application/json" -u admin:district "https://play.dhis2.org/2.38.1.1/api/metadata?importStrategy=UPDATE&mergeMode=MERGE"

After this request is made, you can perform this GET request in a browser:

https://play.dhis2.org/2.38.1.1/api/organisationUnits/ImspTQPwCqd/translations

Which returns:

<organisationUnit xmlns="http://dhis2.org/schema/dxf/2.0">
<translations>
<translation property="NAME" locale="en_GB" value="Sierra Leone"/>
<translation property="NAME" locale="en_GB" value="test merge mode"/>
<translation property="SHORT_NAME" locale="en_GB" value="Sierra Leone"/>
</translations>
</organisationUnit>

Which tells us that the DHIS2 has allowed us to corrupt the database with duplicate values for the NAME + en_GB pair.

The important thing to note is that this only happens when mergeMode is set to MERGE.

For #2 I am referring to the limitations of both the
api/organisationUnits/<id>/translations

endpoint as well as the
api/metadata

endpoint

#3 can be reproduced with the following curl:
curl -X POST -d '{"organisationUnits":[{"id":"ImspTQPwCqd","translations":[{"property":"NAME","value":"test required values","locale":"en_GB"}]}]}' -H "Content-Type: application/json" -u admin:district "https://play.dhis2.org/2.38.1.1/api/metadata?importStrategy=UPDATE&mergeMode=MERGE"

Which, when executed, returns:

{"httpStatus":"Conflict","httpStatusCode":409,"status":"WARNING","message":"One or more errors occurred, please see full details in import report.","response":{"responseType":"ImportReport","stats":{"created":0,"updated":0,"deleted":0,"ignored":1,"total":1},"typeReports":[{"klass":"org.hisp.dhis.organisationunit.OrganisationUnit","stats":{"created":0,"updated":0,"deleted":0,"ignored":1,"total":1},"objectReports":[{"klass":"org.hisp.dhis.organisationunit.OrganisationUnit","index":0,"uid":"ImspTQPwCqd","errorReports":[{"message":"Missing required property name.","mainKlass":"org.hisp.dhis.organisationunit.OrganisationUnit","errorCode":"E4000","errorKlass":"java.lang.String","errorProperty":"name","errorProperties":["name"]},{"message":"Missing required property shortName.","mainKlass":"org.hisp.dhis.organisationunit.OrganisationUnit","errorCode":"E4000","errorKlass":"java.lang.String","errorProperty":"shortName","errorProperties":["shortName"]},{"message":"Missing required property openingDate.","mainKlass":"org.hisp.dhis.organisationunit.OrganisationUnit","errorCode":"E4000","errorKlass":"java.util.Date","errorProperty":"openingDate","errorProperties":["openingDate"]}]}]}],"status":"ERROR"}}

These values should not be required for updating existing data.