Inconsistent behavior between dataValues and dataValueSets endpoints in regards to data set period expiry

Hi Devs,
When data is imported through the API through the dataValues end point, it appears that if a data set which the data elements which are being reported has an period which is locked (or expired) it is not possible to import the data. You get a message like

{“httpStatus”:“Conflict”,“httpStatusCode”:409,“status”:“ERROR”,“message”:“Data set is locked”}

When you import the same data through the dataValueSets endpoint, it works.

In this case, you get

{“responseType”:“ImportSummary”,“status”:“SUCCESS”,“description”:“Import process completed successfully”,“importCount”:{“imported”:21,“updated”:0,“ignored”:0,“deleted”:0},“dataSetComplete”:“false”}

as an example.

This does not seem to be entirely consistent behavior. Is this being considered to be implemented in a future release?

Regards,

Jason

Hi Morten,

Is this thing related to the other discussion we have? I think we should stick to one consistent format - like returning importSummaries always.

···

On Tue, Aug 18, 2015 at 9:13 AM, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Devs,
When data is imported through the API through the dataValues end point, it appears that if a data set which the data elements which are being reported has an period which is locked (or expired) it is not possible to import the data. You get a message like

{“httpStatus”:“Conflict”,“httpStatusCode”:409,“status”:“ERROR”,“message”:“Data set is locked”}

When you import the same data through the dataValueSets endpoint, it works.

In this case, you get

{“responseType”:“ImportSummary”,“status”:“SUCCESS”,“description”:“Import process completed successfully”,“importCount”:{“imported”:21,“updated”:0,“ignored”:0,“deleted”:0},“dataSetComplete”:“false”}

as an example.

This does not seem to be entirely consistent behavior. Is this being considered to be implemented in a future release?

Regards,

Jason


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp


Thank you,

Abyot.

Hm, I think this is something different entirely… If I understand Jason right, its only checking for locked data sets on /api/dataValues and not on /api/dataValueSets ?

···

On Tue, Aug 18, 2015 at 4:21 PM, Abyot Gizaw abyota@gmail.com wrote:

Hi Morten,

Is this thing related to the other discussion we have? I think we should stick to one consistent format - like returning importSummaries always.


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp


Morten


Thank you,

Abyot.

On Tue, Aug 18, 2015 at 9:13 AM, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Devs,
When data is imported through the API through the dataValues end point, it appears that if a data set which the data elements which are being reported has an period which is locked (or expired) it is not possible to import the data. You get a message like

{“httpStatus”:“Conflict”,“httpStatusCode”:409,“status”:“ERROR”,“message”:“Data set is locked”}

When you import the same data through the dataValueSets endpoint, it works.

In this case, you get

{“responseType”:“ImportSummary”,“status”:“SUCCESS”,“description”:“Import process completed successfully”,“importCount”:{“imported”:21,“updated”:0,“ignored”:0,“deleted”:0},“dataSetComplete”:“false”}

as an example.

This does not seem to be entirely consistent behavior. Is this being considered to be implemented in a future release?

Regards,

Jason


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Ok could be. My worry is we should return similar object whether one or many.

···


Morten

On Tue, Aug 18, 2015 at 4:21 PM, Abyot Gizaw abyota@gmail.com wrote:

Hi Morten,

Is this thing related to the other discussion we have? I think we should stick to one consistent format - like returning importSummaries always.


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp


Thank you,

Abyot.

On Tue, Aug 18, 2015 at 9:13 AM, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Devs,
When data is imported through the API through the dataValues end point, it appears that if a data set which the data elements which are being reported has an period which is locked (or expired) it is not possible to import the data. You get a message like

{“httpStatus”:“Conflict”,“httpStatusCode”:409,“status”:“ERROR”,“message”:“Data set is locked”}

When you import the same data through the dataValueSets endpoint, it works.

In this case, you get

{“responseType”:“ImportSummary”,“status”:“SUCCESS”,“description”:“Import process completed successfully”,“importCount”:{“imported”:21,“updated”:0,“ignored”:0,“deleted”:0},“dataSetComplete”:“false”}

as an example.

This does not seem to be entirely consistent behavior. Is this being considered to be implemented in a future release?

Regards,

Jason


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Hi Morten and Abyot,
Yes, I think Abyot’s concern is slightly different, i.e. the structure of the actual response. I am not worried about that at this point, but rather the inconsistent behaviour regarding data import into “locked” periods between the two imports.

Regards,

Jason

···

On Tue, Aug 18, 2015 at 12:02 PM, Abyot Gizaw abyota@gmail.com wrote:

Ok could be. My worry is we should return similar object whether one or many.


Thank you,

Abyot.

(sent from mobile)

On Aug 18, 2015 11:44 AM, “Morten Olav Hansen” mortenoh@gmail.com wrote:

Hm, I think this is something different entirely… If I understand Jason right, its only checking for locked data sets on /api/dataValues and not on /api/dataValueSets ?


Morten

On Tue, Aug 18, 2015 at 4:21 PM, Abyot Gizaw abyota@gmail.com wrote:

Hi Morten,

Is this thing related to the other discussion we have? I think we should stick to one consistent format - like returning importSummaries always.


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp


Thank you,

Abyot.

On Tue, Aug 18, 2015 at 9:13 AM, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Devs,
When data is imported through the API through the dataValues end point, it appears that if a data set which the data elements which are being reported has an period which is locked (or expired) it is not possible to import the data. You get a message like

{“httpStatus”:“Conflict”,“httpStatusCode”:409,“status”:“ERROR”,“message”:“Data set is locked”}

When you import the same data through the dataValueSets endpoint, it works.

In this case, you get

{“responseType”:“ImportSummary”,“status”:“SUCCESS”,“description”:“Import process completed successfully”,“importCount”:{“imported”:21,“updated”:0,“ignored”:0,“deleted”:0},“dataSetComplete”:“false”}

as an example.

This does not seem to be entirely consistent behavior. Is this being considered to be implemented in a future release?

Regards,

Jason


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+46764147049

Jason,

Could you make a bug report and assign it to me? I will have a look tomorrow, are you sending a data value set with datasets, or just a set of data elements?

···

On Tue, Aug 18, 2015 at 5:04 PM, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Morten and Abyot,
Yes, I think Abyot’s concern is slightly different, i.e. the structure of the actual response. I am not worried about that at this point, but rather the inconsistent behaviour regarding data import into “locked” periods between the two imports.

Regards,

Jason


Morten

On Tue, Aug 18, 2015 at 12:02 PM, Abyot Gizaw abyota@gmail.com wrote:

Ok could be. My worry is we should return similar object whether one or many.


Thank you,

Abyot.

(sent from mobile)

On Aug 18, 2015 11:44 AM, “Morten Olav Hansen” mortenoh@gmail.com wrote:

Hm, I think this is something different entirely… If I understand Jason right, its only checking for locked data sets on /api/dataValues and not on /api/dataValueSets ?

Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+46764147049


Morten

On Tue, Aug 18, 2015 at 4:21 PM, Abyot Gizaw abyota@gmail.com wrote:

Hi Morten,

Is this thing related to the other discussion we have? I think we should stick to one consistent format - like returning importSummaries always.


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp


Thank you,

Abyot.

On Tue, Aug 18, 2015 at 9:13 AM, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Devs,
When data is imported through the API through the dataValues end point, it appears that if a data set which the data elements which are being reported has an period which is locked (or expired) it is not possible to import the data. You get a message like

{“httpStatus”:“Conflict”,“httpStatusCode”:409,“status”:“ERROR”,“message”:“Data set is locked”}

When you import the same data through the dataValueSets endpoint, it works.

In this case, you get

{“responseType”:“ImportSummary”,“status”:“SUCCESS”,“description”:“Import process completed successfully”,“importCount”:{“imported”:21,“updated”:0,“ignored”:0,“deleted”:0},“dataSetComplete”:“false”}

as an example.

This does not seem to be entirely consistent behavior. Is this being considered to be implemented in a future release?

Regards,

Jason


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Hi Jason,

yes this is not consistently implemented. The reason is simply performance - checking for approval for a large batch of data values will be costly. We plan to fix this properly.

regards,

Lars

···

On Tue, Aug 18, 2015 at 6:29 AM, Morten Olav Hansen mortenoh@gmail.com wrote:

Jason,

Could you make a bug report and assign it to me? I will have a look tomorrow, are you sending a data value set with datasets, or just a set of data elements?


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp


Morten

On Tue, Aug 18, 2015 at 5:04 PM, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Morten and Abyot,
Yes, I think Abyot’s concern is slightly different, i.e. the structure of the actual response. I am not worried about that at this point, but rather the inconsistent behaviour regarding data import into “locked” periods between the two imports.

Regards,

Jason

On Tue, Aug 18, 2015 at 12:02 PM, Abyot Gizaw abyota@gmail.com wrote:

Ok could be. My worry is we should return similar object whether one or many.


Thank you,

Abyot.

(sent from mobile)

On Aug 18, 2015 11:44 AM, “Morten Olav Hansen” mortenoh@gmail.com wrote:

Hm, I think this is something different entirely… If I understand Jason right, its only checking for locked data sets on /api/dataValues and not on /api/dataValueSets ?

Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+46764147049


Morten

On Tue, Aug 18, 2015 at 4:21 PM, Abyot Gizaw abyota@gmail.com wrote:

Hi Morten,

Is this thing related to the other discussion we have? I think we should stick to one consistent format - like returning importSummaries always.


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp


Thank you,

Abyot.

On Tue, Aug 18, 2015 at 9:13 AM, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Devs,
When data is imported through the API through the dataValues end point, it appears that if a data set which the data elements which are being reported has an period which is locked (or expired) it is not possible to import the data. You get a message like

{“httpStatus”:“Conflict”,“httpStatusCode”:409,“status”:“ERROR”,“message”:“Data set is locked”}

When you import the same data through the dataValueSets endpoint, it works.

In this case, you get

{“responseType”:“ImportSummary”,“status”:“SUCCESS”,“description”:“Import process completed successfully”,“importCount”:{“imported”:21,“updated”:0,“ignored”:0,“deleted”:0},“dataSetComplete”:“false”}

as an example.

This does not seem to be entirely consistent behavior. Is this being considered to be implemented in a future release?

Regards,

Jason


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Lars Helge Øverland

Lead developer, DHIS 2

University of Oslo

Skype: larshelgeoverland

http://www.dhis2.org

Hi Lars,
Sounds good. Specifically, this bug report was related to locked periods, but I suppose it is the same for other dimensions (approvals, cat combos, attribute combos, data elements and orgunits)

I have created the bug report here, but it really may be more of a feature request? :slight_smile:

https://bugs.launchpad.net/dhis2/+bug/1485973

Regards,

Jason

···

On Tue, Aug 18, 2015 at 1:53 PM, Lars Helge Øverland larshelge@gmail.com wrote:

Hi Jason,

yes this is not consistently implemented. The reason is simply performance - checking for approval for a large batch of data values will be costly. We plan to fix this properly.

regards,

Lars

On Tue, Aug 18, 2015 at 6:29 AM, Morten Olav Hansen mortenoh@gmail.com wrote:

Jason,

Could you make a bug report and assign it to me? I will have a look tomorrow, are you sending a data value set with datasets, or just a set of data elements?


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Lars Helge Øverland

Lead developer, DHIS 2

University of Oslo

Skype: larshelgeoverland

http://www.dhis2.org


Morten

On Tue, Aug 18, 2015 at 5:04 PM, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Morten and Abyot,
Yes, I think Abyot’s concern is slightly different, i.e. the structure of the actual response. I am not worried about that at this point, but rather the inconsistent behaviour regarding data import into “locked” periods between the two imports.

Regards,

Jason

On Tue, Aug 18, 2015 at 12:02 PM, Abyot Gizaw abyota@gmail.com wrote:

Ok could be. My worry is we should return similar object whether one or many.


Thank you,

Abyot.

(sent from mobile)

On Aug 18, 2015 11:44 AM, “Morten Olav Hansen” mortenoh@gmail.com wrote:

Hm, I think this is something different entirely… If I understand Jason right, its only checking for locked data sets on /api/dataValues and not on /api/dataValueSets ?

Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+46764147049


Morten

On Tue, Aug 18, 2015 at 4:21 PM, Abyot Gizaw abyota@gmail.com wrote:

Hi Morten,

Is this thing related to the other discussion we have? I think we should stick to one consistent format - like returning importSummaries always.


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp


Thank you,

Abyot.

On Tue, Aug 18, 2015 at 9:13 AM, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Devs,
When data is imported through the API through the dataValues end point, it appears that if a data set which the data elements which are being reported has an period which is locked (or expired) it is not possible to import the data. You get a message like

{“httpStatus”:“Conflict”,“httpStatusCode”:409,“status”:“ERROR”,“message”:“Data set is locked”}

When you import the same data through the dataValueSets endpoint, it works.

In this case, you get

{“responseType”:“ImportSummary”,“status”:“SUCCESS”,“description”:“Import process completed successfully”,“importCount”:{“imported”:21,“updated”:0,“ignored”:0,“deleted”:0},“dataSetComplete”:“false”}

as an example.

This does not seem to be entirely consistent behavior. Is this being considered to be implemented in a future release?

Regards,

Jason


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+46764147049

Thanks for the report. It is closer to a bug than a feature.

regards,

Lars

···

On Tue, Aug 18, 2015 at 8:09 AM, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Lars,
Sounds good. Specifically, this bug report was related to locked periods, but I suppose it is the same for other dimensions (approvals, cat combos, attribute combos, data elements and orgunits)

I have created the bug report here, but it really may be more of a feature request? :slight_smile:

https://bugs.launchpad.net/dhis2/+bug/1485973

Regards,

Jason

On Tue, Aug 18, 2015 at 1:53 PM, Lars Helge Øverland larshelge@gmail.com wrote:

Hi Jason,

yes this is not consistently implemented. The reason is simply performance - checking for approval for a large batch of data values will be costly. We plan to fix this properly.

regards,

Lars


Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+46764147049

On Tue, Aug 18, 2015 at 6:29 AM, Morten Olav Hansen mortenoh@gmail.com wrote:

Jason,

Could you make a bug report and assign it to me? I will have a look tomorrow, are you sending a data value set with datasets, or just a set of data elements?


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Lars Helge Øverland

Lead developer, DHIS 2

University of Oslo

Skype: larshelgeoverland

http://www.dhis2.org


Morten

On Tue, Aug 18, 2015 at 5:04 PM, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Morten and Abyot,
Yes, I think Abyot’s concern is slightly different, i.e. the structure of the actual response. I am not worried about that at this point, but rather the inconsistent behaviour regarding data import into “locked” periods between the two imports.

Regards,

Jason

On Tue, Aug 18, 2015 at 12:02 PM, Abyot Gizaw abyota@gmail.com wrote:

Ok could be. My worry is we should return similar object whether one or many.


Thank you,

Abyot.

(sent from mobile)

On Aug 18, 2015 11:44 AM, “Morten Olav Hansen” mortenoh@gmail.com wrote:

Hm, I think this is something different entirely… If I understand Jason right, its only checking for locked data sets on /api/dataValues and not on /api/dataValueSets ?

Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+46764147049


Morten

On Tue, Aug 18, 2015 at 4:21 PM, Abyot Gizaw abyota@gmail.com wrote:

Hi Morten,

Is this thing related to the other discussion we have? I think we should stick to one consistent format - like returning importSummaries always.


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp


Thank you,

Abyot.

On Tue, Aug 18, 2015 at 9:13 AM, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Devs,
When data is imported through the API through the dataValues end point, it appears that if a data set which the data elements which are being reported has an period which is locked (or expired) it is not possible to import the data. You get a message like

{“httpStatus”:“Conflict”,“httpStatusCode”:409,“status”:“ERROR”,“message”:“Data set is locked”}

When you import the same data through the dataValueSets endpoint, it works.

In this case, you get

{“responseType”:“ImportSummary”,“status”:“SUCCESS”,“description”:“Import process completed successfully”,“importCount”:{“imported”:21,“updated”:0,“ignored”:0,“deleted”:0},“dataSetComplete”:“false”}

as an example.

This does not seem to be entirely consistent behavior. Is this being considered to be implemented in a future release?

Regards,

Jason


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Lars Helge Øverland

Lead developer, DHIS 2

University of Oslo

Skype: larshelgeoverland

http://www.dhis2.org