Android Capture and SMS

Hey @chase.freeman . I had the same issue and have been discussing with @Jaime as there might be something wrong with the gateway. In any case, the why I solved it was stopping the app (via settings > force stop) and removing the send SMS permission. Open it again and configure with URL, username and password (make sure the user has a phone number in its specific section of the user config and it matches the SMS sender) then try again and you should see things happening in the logs.

In any case, we will review the app here: [ANDROAPP-4864] - Jira

1 Like

Cool thanks @jaime.bosque - I’ll give it a shot tomorrow.

1 Like

@jaime.bosque - I’m back on SMS :grin:
I’ve had success with the SMS sync using the Android DHIS2 SMS Gateway. The SMS generated by the Android App (v2.7.1.1) is being received on the Server. However, I don’t know what to do with with now! :man_shrugging:

Can you please advise on next steps to unencrypt the message and have it available in the Capture app along with other events? Do I need to setup a Command/parser to handle this?

1 Like

Hi @chase.freeman

Please, remember that the Android DHIS2 SMS Gateway is something that we DO NOT recommed using in production and that in those cases you should use a real gateway. In anycase, if you are receving the SMS in the backend everything should be automatically handled by the backend module.

I will provide an example for a DataSet with detailed information on every step (so this can be used as further reference post for other people).

  1. We have the DHIS2 Android App without Internet connectivity and we want to synchronize with SMS. For this we need to make sure that the SMS Gateway has been established in the phone either via the Android Settings Web App or manually in the settings (will not go into that as this is already well documented).

  2. Depending on the version of the App the workflow might differ but you will end up sending an SMS like below, this can always be reviewed in the SMS application of the phone

  3. This SMS is received by the Gateway, here you have several options. You are claiming to use the Android DHIS2 SMS Gateway App. It doesn’t matter, the important thing is to note that the Gateway will generate an API request like the following (to simplify I am showing/performing it manually)

Please note in the picture above how the text value corresponds to the SMS sent by the phone. Also, important to make sure that the user that is making the request has a matching phone number (see common issues further below). In this case I am encoding admin / district with basic auth. I am showing this here:

  1. If the API POST request is properly composed and the values mentioned above are correct, the server will reply with a 200.

  2. By inspecting the logs, we see how the encoded SMS is understood by the server and the actions performed (in this case, the update of the values in the dataset) without having to do anything else.

INFO  org.hisp.dhis.sms.SmsConsumerThread [taskScheduler-20] Received SMS: sample text
WARN  org.hisp.dhis.sms.SmsConsumerThread [taskScheduler-20] No SMS command found in received data
INFO  org.hisp.dhis.sms.SmsConsumerThread [taskScheduler-9] Received SMS: pAJkAHDBAV+4GjmtpUlUVTl5PTUDmud9iohJJxUyZGBkZmBiAZyggQPXNMBq1PQL0UDgf39qCA==
INFO  org.hisp.dhis.sms.listener.CompressionSMSListener [taskScheduler-9] New received Sms submission decoded as: {
  "orgUnit": {
    "uid": "DiszpKrYNg8",
    "hash": 941647,
    "type": "ORGANISATION_UNIT"
  },
  "dataSet": {
    "uid": "ce7DSxx5H2I",
    "hash": 0,
    "type": "DATASET"
  },
  "complete": false,
  "attributeOptionCombo": {
    "uid": "HllvX50cXC0",
    "hash": 2713,
    "type": "CATEGORY_OPTION_COMBO"
  },
  "period": "202301",
  "values": [
    {
      "categoryOptionCombo": {
        "uid": "sqGRzCziswD",
        "hash": 129,
        "type": "CATEGORY_OPTION_COMBO"
      },
      "dataElement": {
        "uid": "FTRrcoaog83",
        "hash": 965030,
        "type": "DATA_ELEMENT"
      },
      "value": "1",
      "smsValue": {
        "value": 1,
        "type": "INT"
      }
    },
    {
      "categoryOptionCombo": {
        "uid": "sqGRzCziswD",
        "hash": 129,
        "type": "CATEGORY_OPTION_COMBO"
      },
      "dataElement": {
        "uid": "LjNlMTl9Nq9",
        "hash": 354938,
        "type": "DATA_ELEMENT"
      },
      "value": "2",
      "smsValue": {
        "value": 2,
        "type": "INT"
      }
    },
    {
      "categoryOptionCombo": {
        "uid": "sqGRzCziswD",
        "hash": 129,
        "type": "CATEGORY_OPTION_COMBO"
      },
      "dataElement": {
        "uid": "eY5ehpbEsB7",
        "hash": 952839,
        "type": "DATA_ELEMENT"
      },
      "value": "1",
      "smsValue": {
        "value": 1,
        "type": "INT"
      }
    },
    {
      "categoryOptionCombo": {
        "uid": "sqGRzCziswD",
        "hash": 129,
        "type": "CATEGORY_OPTION_COMBO"
      },
      "dataElement": {
        "uid": "Ix2HsbDMLea",
        "hash": 1032117,
        "type": "DATA_ELEMENT"
      },
      "value": "2",
      "smsValue": {
        "value": 2,
        "type": "INT"
      }
    }
  ],
  "header": {
    "type": "AGGREGATE_DATASET",
    "version": 2,
    "lastSyncDate": "Mar 2, 2023, 9:47:45 AM",
    "submissionId": 1
  },
  "userId": {
    "uid": "lt0qSrjAagY",
    "hash": 0,
    "type": "USER"
  }
}
  1. And in fact, the SMS in the inbox can be see marked as processed

  2. We can see how the data matches Data Entry in web (left) an phone (right):

Common issues

  1. In case the user sending the API request doesn’t have a matching phone number (either because of not set or because of mismatch) the API will produce a 409 error
  2. If the SMS sent is not properly formated the API will accept the SMS but will not process it and will mark it as unhandled (this is visible in the logs and in the SMS inbox). Note how I have prepend a “XX” to the SMS sent:

API Request

Error log

INFO org.hisp.dhis.sms.SmsConsumerThread [taskScheduler-19] Received SMS: XXpAJkAHDBAV+4GjmtpUlUVTl5PTUDmud9iohJJxUyZGBkZmBiAZyggQPXNMBq1PQL0UDgf39qCA== 
WARN org.hisp.dhis.sms.SmsConsumerThread [taskScheduler-19] No SMS command found in received data 

SMS Inbox

I hope this clarifies the situation. Let me know if you need further assistance

Excellent @jaime.bosque - thank you. I do not intend to recommend our projects/clients use the dhis2 android gateway - I fully understand the limitations at scale. I am just trying to confirm the process before choosing a gateway providers (either “global” or in country).

Now, I am happy to see there are no extra steps needed when the server receives the SMS, however it appears I am suffering from issue number two but with errors that contradict what you’ve explained – the SMS is received but “unhandled” with the error “No SMS Command Found in Received Data”.

For context: this is for an event with a single data element in a program. Android v. 2.7.1.1 and DHIS2 2.39.0.1

Mar  1 16:07:31 1b74540f server: * INFO  2023-03-01T16:07:31,095 Received SMS: 30Jj/2zQCRkeniIe488ilvoEhawZg1LuoJ4/jmfBFj/tvQMEILHRlc3QgMyAtIEkgc2V0dXAgdGhlIGdhdGV3YXkgYW5kIHRoZSBudW1iZXIgaW4gYW5kcm9pZCBzZXR0aW5ncy4gSSBoYXZlIGVuYWJs (SmsConsumerThread.java [taskScheduler-20])
Mar  1 16:07:31 1b74540f server: * WARN  2023-03-01T16:07:31,103 No SMS command found in received data (SmsConsumerThread.java [taskScheduler-20])

I will be out of the office until Monday but will check any reply then.

Hey @chase.freeman .

I made another example with a Tracker Program sending a singe event and I managed to get a proper response to the server.

INFO org.hisp.dhis.sms.SmsConsumerThread [taskScheduler-9] Received SMS: jSJkAHDBA1+4GjmtpUlUVT8KOkfsgcIT0YT3ntwQUrLAnAtKmCe6rH1YHgWPuARxnhAAYjMrawtjKAZXBE1pY2hlbGxlAJoRiQwtzm3twA1Pwo4UAGtyijlVa2ZxUyKbzhul4dVBYsL+P+Fj7WPwZyllTMtBODnTL80xSZXBsYWNlbWVudADLdYgB4lHS78yK8AAWFrOie6NDK5OYAA=
INFO org.hisp.dhis.sms.listener.CompressionSMSListener [taskScheduler-9] New received Sms submission decoded as: {
“orgUnit”: {
“uid”: “g8upMTyEZGZ”,
“hash”: 1032846,
“type”: “ORGANISATION_UNIT”
},
“trackerProgram”: {
“uid”: “IpHINAT79UW”,
“hash”: 253,
“type”: “PROGRAM”
},
“trackedEntityType”: {
“uid”: “nEenWmSyUEp”,
“hash”: 28,
“type”: “TRACKED_ENTITY_TYPE”
},
“trackedEntityInstance”: {
“uid”: “GdemdlFRWWf”,
“hash”: 0,
“type”: “TRACKED_ENTITY_INSTANCE”
},
“enrollment”: {
“uid”: “ii2S2qgO9xg”,
“hash”: 0,
“type”: “ENROLLMENT”
},
“enrollmentDate”: “Feb 13, 2023, 11:00:00 PM”,
“enrollmentStatus”: “ACTIVE”,
“incidentDate”: “Feb 16, 2023, 10:10:36 AM”,
“values”: [
{
“attribute”: {
“uid”: “cejWyOfXge6”,
“hash”: 16385,
“type”: “TRACKED_ENTITY_ATTRIBUTE”
},
“value”: “Female”,
“smsValue”: {
“value”: “Female”,
“type”: “STRING”
}
},
[…]

To be honest I don’t know what can be going on your side. My first thought is from experience seeing network providers changing/manipulating the SMS so you could probably go to the Android SMS App and check if the string you are pasting here is exactly the same as the one received in the server.

Please check and let me know.

@jaime.bosque - I’m confirming that the server’s log is showing exactly the same value for the message as the DHIS2 Gateway, as well as the SMS config > Received UI.

What Version of DHIS2 are you using?

I DM you to provide you access to the instance if that would be helpful/useful. It is a dev/sandbox environment.

Also - when I try to manually create the API Request (in Postman) I get an HTML response telling me to use the parameter “message” instead of text. But when I do I get the 409 for the user’s phone number not being registered even though it is (the message is received via the android gateway app with same #, though it remains unprocessed).

I performed my tests over 2.39.1.1.

What you need to make sure is that the original SMS is the same (bypass the gateway if you want). This is because at network level operators can mess up with the SMS (sometimes they translate characters and/or include parts to identify the concatenation).

I am not sure about the “message” vs. “text” parameter. Will try to get someone from backend to look at it.

@zubair could you provide some support here? To my understanding this would not be a problem with Android itself but with the SMS library?

@chase.freeman , you can give me access if you want, but there won’t be much I can do without having access to the logs. Not sure if this is available via any web interface wihouth having SSh access to the server…