Use of square brackets [] in metadata field filter

Hi DHIS2 devs,

I faced an issue recently with using (unencoded) square brackets in requests to the DHIS2 API. It turns out it was related to a new version of tomcat (7.0.90). I’ve spoken to the tomcat devs and they’ve said that they will now consider unencoded square brackets in a request to be an error.

According to the docs: https://docs.dhis2.org/master/en/developer/html/webapi_metadata_field_filter.html square brackets are used to indicate a subfield, when using the field query parameter.

Might it be worth considering changing the symbol used for this? At the very least it should be made clear that the requests using it will need to be URL encoded.

There are even requests in the DHIS2 frontend (I’m running v2.27) which have unencoded square brackets. The approvals page in the report view fails with the new version of tomcat for this reason. It would be worth ensuring all frontend requests are properly encoded.

Cheers,

  • Jasper
···

Jasper Timm

Lead Software Developer

image

eHealth Africa
4A Renner Drive
Off Wilkinson Road (behind Monoprix Supermarket)
Freetown, Sierra Leone

mobile +232 99891119

image

We faced the same issue, and I have brought it to Lars’ attention, and he is aware of the issue. According to him, DHIS2 will be addressing this by changing square brackets in the long term, but in the short term solution you can make square brackets be exempted on the tomcat side by using relaxedQueryChars=“” in the Tomcat connector configuration.

···

On Tue, Oct 30, 2018 at 6:57 AM Jasper Timm jasper.timm@ehealthafrica.org wrote:

Hi DHIS2 devs,

I faced an issue recently with using (unencoded) square brackets in requests to the DHIS2 API. It turns out it was related to a new version of tomcat (7.0.90). I’ve spoken to the tomcat devs and they’ve said that they will now consider unencoded square brackets in a request to be an error.

According to the docs: https://docs.dhis2.org/master/en/developer/html/webapi_metadata_field_filter.html square brackets are used to indicate a subfield, when using the field query parameter.

Might it be worth considering changing the symbol used for this? At the very least it should be made clear that the requests using it will need to be URL encoded.

There are even requests in the DHIS2 frontend (I’m running v2.27) which have unencoded square brackets. The approvals page in the report view fails with the new version of tomcat for this reason. It would be worth ensuring all frontend requests are properly encoded.

Cheers,

  • Jasper


Jasper Timm

Lead Software Developer

eHealth Africa
4A Renner Drive
Off Wilkinson Road (behind Monoprix Supermarket)
Freetown, Sierra Leone

mobile +232 99891119


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Hi there,

Vlad is right on the above.

Longer term we will work on replacing the use of square brackets with a URL safe character (there are not many left) and/or perhaps a POST-with-JSON-payload based solution.

In the short term we will ensure all web API requests are encoded.

As Vlad say, the best option short term is to use r****elaxedQueryChars=“[]” at the Tomcat HTTP connector level.

Jasper: Are you able to tell us in which apps and which requests had those unencoded requests?

best,

Lars

···

Lars Helge Øverland

Technical lead, DHIS 2

University of Oslo

lars@dhis2.org

https://www.dhis2.org

Thanks guys.

@Lars - (This is v2.27) On the Reports page, if you go to Data Approvals (i.e. dhis-web-reporting/showDataApprovalForm.action), I see a request to the following URL:

api/dataSets.json?fields=id,displayName,periodType,workflow,categoryCombo[id,displayName]&paging=false

Cheers,

  • Jasper
···

Jasper Timm

Lead Software Developer

image

eHealth Africa
4A Renner Drive
Off Wilkinson Road (behind Monoprix Supermarket)
Freetown, Sierra Leone

mobile +232 99891119

image