Updating tracked entities - Inconsistencies between API and documentation?

Support - Assistance technique
,
1

Related to Update strategies for Tracked Entity Instances. My use case is that I am uploading historical data to DHIS2 via the web API. I would like to create TEIs if the record does not already exist in DHIS2, but update the TEI record if a matching record is found based on the identifier I specify (e.g., farmerID).

The DHIS2 documentation suggests that I can set an identifier (e.g., farmerID) for a tracked entity when posting TEI data to DHIS2.

From the web api documentation:

1.67.5 Update strategies. Two update strategies for tracked entity instance are supported: enrollment and event creation. This is useful when you have generated an identifier on the client side and are not sure if it was created or not on the server.
CREATE_AND_UPDATE: Try and match the ID, if it exist then update, if not create.

I have been testing with DHIS 2.29 using this CREATE_AND_UPDATE strategy in my post request to DHIS2, and the response will always try to create a new tracked entity (but never update if it already exists).

curl -X PUT -H "Content-Type:application/json" -d @payload.json -u $UN:$PW https://severname/api/29/trackedEntityInstances?strategy=CREATE_AND UPDATE

This CREATE_AND_UPDATE strategy will only update a tracked entity if I include the TrackedEntityInstance (tei uid) generated by DHIS2 in my payload. Therefore can I not generate a new/ use a different external unique identifier (farmerID) to lookup and update existing tracked entity instances? Must I always use the DHIS2-generated tracked entity instance ID?

To sum it up, we have generated an ID ( farmerID ) on the client side, and we’re not sure if it was created or not on the server. This is the classic case for an upsert (i.e., strategy=CREATE_AND UPDATE ) and is reflected in the documentation, but the current behavior of the API suggests that client-side generated IDs can not be used for matching on upserts. If that’s correct (only server-generated IDs can be used), how does one perform the upsert given that the client-side system won’t know the server-generated ID if it hasn’t yet been created?

8 Likes
4

Hi Aleksa,

before replying further, how did you model your external “Farmer ID”? Did you use the code property or did you use a tracked entity attribute? If the latter, did you mark the attribute as unique?

1 Like
5

Hi Aleksa,

Currently, the DHIS2 Tracker endpoints work only with UIDs. We check whether given UID is already in the system and if yes, we do update it. If it is not in the system, we create it.

Unfortunately, we do not support this “update if present/create if not present” logic for other IDs than UID. The documentation is a bit ambiguous in this and should/will be improved.

So I am sorry to say that your use case won’t work. However, there is one option how to at least find what entities are already present in the system. If this “farmer id” is an attribute, then you can mark it as unique (in Maintenance app → Attribute → details of given attribute). When some attribute is marked as unique, the system checks for uniqueness of that attribute on each insert/update. If a duplicate appears, an error is generated. Otherwise it is successfully stored. With this approach you should be able to find a duplicates and then manually “merge” them. It requires some manual work but this is the best solution to your problem I can offer.

1 Like
6

Sorry for the big delay! @Lars, it was a tracked entity attribute and it was marked as unique.

@David_Katuscak, we’re looking at these two bits of the dhis2 API docs: webapi_identifier_schemes and webapi_tei_update_strategies.

The latter says, “Two update strategies for all 3 tracker endpoints are supported: enrollment and event creation. This is useful when you have generated an identifier on the client side and are not sure if it was created or not on the server.” :thinking: I’m guessing this is what we’re somehow misunderstanding!

In the former, trackedEntityIdScheme is one of the examples provided, and the docs show that to use this scheme you’ll need to append &trackedEntityIdScheme=ATTRIBUTE=<the attribute ID>.

So… the request we thought would work was to: /2.35.0/api/trackedEntityInstances?strategy=CREATE_AND_UPDATE&trackedEntityIdScheme=ATTRIBUTE%3AX4b6BdLIDOr

Strangely, the resulting 409 conflict from DHIS2 specifies that there are no idSchemes in use ("idSchemes":{}) while "importStrategy":"CREATE_AND_UPDATE".

Any thoughts on this? Even if this is not possible, it would be great to see a working example of some way to use that strategy=CREATE_AND_UPDATE for the tracked entity instance API. I’ve tried using the DHIS2 generated UUID as well and get no love.

My colleague will post the exact body of the requests we’re using right now. Hope that’s helpful! FWIW, using code doesn’t seem to work either… but again, if we’ve misinterpreted what is and isn’t possible with tracker that would explain it :slight_smile:

7

Hi @Lars and @David_Katuscak ,
How are Identifier Schemes: trackedEntityIdScheme and trackedEntityAttributeIdScheme supposed to be used? According to the documentation here, it seems it is possible to match on an entity’s attribute or CODE but both id schemes are not recognised and the request fails in both cases:

A. Matching on CODE of TrackedEntityInstance

  1. Create Entity Instance With CODE as Id Scheme
POST  https://play.dhis2.org/2.35.0/api/trackedEntityInstances?strategy=CREATE_AND_UPDATE&trackedEntityIdScheme=CODE
{
"trackedEntityType":"nEenWmSyUEp",
"orgUnit":"g8upMTyEZGZ",
"code":456,
"attributes":[
{"attribute":"w75KJ2mc4zz","value":"Chaiwa"},{"attribute":"zDhUuAYrxNC","value":"NONONONO"},{"attribute":"X4b6BdLIDOr","value":"789"}
]
}
  1. Attempt to Upsert the above created record, but the request fails! We expect it to update the record since we have sent the same code in the request, and the docs say DHIS2 will match on CODE if trackedEntityIdScheme=CODE:
POST  https://play.dhis2.org/2.35.0/api/trackedEntityInstances?strategy=CREATE_AND_UPDATE&trackedEntityIdScheme=CODE
{
"trackedEntityType":"nEenWmSyUEp",
"orgUnit":"g8upMTyEZGZ",
"code":456,
"attributes": [
{"attribute":"w75KJ2mc4zz","value":"Test New Value"},{"attribute":"zDhUuAYrxNC","value":"NONONONO"},{"attribute":"X4b6BdLIDOr","value":"789"}
]}

B. Matching on default UID(id) of TrackedEntityInstance

  1. Create Entity Instance With Default IdSchemes
POST https://play.dhis2.org/2.35.0/api/trackedEntityInstances?strategy=CREATE_AND_UPDATE
{
"trackedEntityType":"nEenWmSyUEp",
"orgUnit":"g8upMTyEZGZ",
"attributes":[
{"attribute":"w75KJ2mc4zz","value":"James"},{"attribute":"zDhUuAYrxNC","value":"NONONONO"},{"attribute":"X4b6BdLIDOr","value":"788"}
]
}
  1. Attempt to Upsert the above created record on UID(id)(we use the uid generated by DHIS2 in step 1), but the request fails! We expect it to update the record since we have sent the same id in the request, and the docs say DHIS2 will match on id by default.
POST https://play.dhis2.org/2.35.0/api/trackedEntityInstances?strategy=CREATE_AND_UPDATE

{
"trackedEntityType":"nEenWmSyUEp",
"orgUnit":"g8upMTyEZGZ",
"id":"kopmlYmVRZl",
"attributes":[
{"attribute":"w75KJ2mc4zz","value":"Test New Value"},{"attribute":"zDhUuAYrxNC","value":"NONONONO"},{"attribute":"X4b6BdLIDOr","value":"788"}
]}

C. Matching on Attribute of TrackedEntityInstance

  1. Create Entity Instance With Unique Attribute, Using trackedEntityIdScheme
POST https://play.dhis2.org/2.35.0/api/trackedEntityInstances?strategy=CREATE_AND_UPDATE&trackedEntityIdScheme=ATTRIBUTE:X4b6BdLIDOr
{
"trackedEntityType":"nEenWmSyUEp",
"orgUnit":"g8upMTyEZGZ",
"attributes":[
{"attribute":"w75KJ2mc4zz","value":"James"},{"attribute":"zDhUuAYrxNC","value":"NONONONO"},{"attribute":"X4b6BdLIDOr","value":"788"}
]
}
  1. Attempt to Upsert the above created record on ATTRIBUTE(we use a unique attribute of our entity), but the request fails! We expect it to update the record since we have sent the same value for the X4b6BdLIDOr attribute in the request, and the docs say DHIS2 will match on ATTRIBUTE if the id scheme is configured like this: trackedEntityIdScheme=ATTRIBUTE:X4b6BdLIDOr 
POST https://play.dhis2.org/2.35.0/api/trackedEntityInstances?strategy=CREATE_AND_UPDATE&trackedEntityIdScheme=ATTRIBUTE:X4b6BdLIDOr

{
"trackedEntityType":"nEenWmSyUEp",
"orgUnit":"g8upMTyEZGZ",
"attributes":[
{"attribute":"w75KJ2mc4zz","value":"Test New Value"},{"attribute":"zDhUuAYrxNC","value":"NONONONO"},{"attribute":"X4b6BdLIDOr","value":"788"}
]}

In summary, we have two main questions:

  1. What Id schemes are supported for the trackedEntityInstance endpoints?
  2. Would you provide a working example of using the CREATE_AND_UPDATE strategy in DHIS2? It hasn’t worked for us yet!
1 Like
8

Hey all, I just bumped into this again while doing bulk TEI syncs → any idea on if it’s possible or how to use trackedEntityIdScheme: 'ATTRIBUTE:rDeWj9yYtzv' when trying to create_and_update a big array of tracked entity instances?

4 Likes