Reserving unique Case ID attributes in Tracker - duplicates encountered when reserving via Android

Hi,

In 2.28. the table “TrackedEntityAttributeReservedValues” is storing unique ID values that have been reserved. When a request is coming from e.g. an Android device running Tracker Capture, the API will trigger the reservation of 100 values and return these to the device while also storing the reserved values in the above table.

Yesterday, I inserted a list of such reserved values into that table in order to “protect” a section of numbers that will be used for importing legacy data.

I then decided to verify that those case IDs WERE now actually reserved.

when using the normal browser Tracker Capture, it clearly worked - no case IDs coming up were in the reserved range

But when I fired up Android Tracker Capture and logged in afresh, the system did reserve 100 more values BUT a number of those were values already in the TrackedEntityAttributeReservedValues table.

Or in other words, it seems like a request for reserved values from Tracker Capture via the API will generate reserved values without checking if those values are reserved already - is that correct?

Regards

Calle

···

Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg


Hey Calle,

we haven’t really documented how this behaves when you do manual inserts of reseved values. Assuming the db insert did the same as a API reservation would have done, and that no caching is interfering what hibernate sees - the 100 new reserved values should not be among those preexisting in the TrackedEntityAttributeReservedValues.

Markus

···

Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg


Markus,

I think this must have been caused by a hibernate caching delay. I deleted the additional records created by Tracker Capture now (after logging out from Tracker Capture), and then logged in with TC again - and this time the system actually generated 200 reserved values in two batches:

  • the first batch of 100 reserved values were created within 20 seconds

  • after 3 minutes, another batch of 100 reserved values were created, and this also took less than 20 sec

AND, more importantly, none of the 200 new values were among those pre-existing in the …ReservedValues table.

So my conclusion is that the only safe route after inserting reservedvalues directly is to re-start the server and thus make sure all caches are flushed BEFORE anybody can log in from a remote device.

Regards

Calle

···

On 19 April 2018 at 14:25, Markus Bekken markus@dhis2.org wrote:

Hey Calle,
we haven’t really documented how this behaves when you do manual inserts of reseved values. Assuming the db insert did the same as a API reservation would have done, and that no caching is interfering what hibernate sees - the 100 new reserved values should not be among those preexisting in the TrackedEntityAttributeReservedValues.

Markus

  1. apr. 2018 kl. 13:26 skrev Calle Hedberg calle.hedberg@gmail.com:

Hi,

In 2.28. the table “TrackedEntityAttributeReservedValues” is storing unique ID values that have been reserved. When a request is coming from e.g. an Android device running Tracker Capture, the API will trigger the reservation of 100 values and return these to the device while also storing the reserved values in the above table.

Yesterday, I inserted a list of such reserved values into that table in order to “protect” a section of numbers that will be used for importing legacy data.

I then decided to verify that those case IDs WERE now actually reserved.

when using the normal browser Tracker Capture, it clearly worked - no case IDs coming up were in the reserved range

But when I fired up Android Tracker Capture and logged in afresh, the system did reserve 100 more values BUT a number of those were values already in the TrackedEntityAttributeReservedValues table.

Or in other words, it seems like a request for reserved values from Tracker Capture via the API will generate reserved values without checking if those values are reserved already - is that correct?

Regards

Calle


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg



Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help : https://help.launchpad.net/ListHelp


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg


Hi

Note also that Tracker Capture did not open up before both 100 reserved value batches had been created - so it took around 4 min for the login

Regards

Calle

···

On 19 April 2018 at 16:13, Calle Hedberg calle.hedberg@gmail.com wrote:

Markus,

I think this must have been caused by a hibernate caching delay. I deleted the additional records created by Tracker Capture now (after logging out from Tracker Capture), and then logged in with TC again - and this time the system actually generated 200 reserved values in two batches:

  • the first batch of 100 reserved values were created within 20 seconds
  • after 3 minutes, another batch of 100 reserved values were created, and this also took less than 20 sec

AND, more importantly, none of the 200 new values were among those pre-existing in the …ReservedValues table.

So my conclusion is that the only safe route after inserting reservedvalues directly is to re-start the server and thus make sure all caches are flushed BEFORE anybody can log in from a remote device.

Regards

Calle

On 19 April 2018 at 14:25, Markus Bekken markus@dhis2.org wrote:

Hey Calle,
we haven’t really documented how this behaves when you do manual inserts of reseved values. Assuming the db insert did the same as a API reservation would have done, and that no caching is interfering what hibernate sees - the 100 new reserved values should not be among those preexisting in the TrackedEntityAttributeReservedValues.

Markus

  1. apr. 2018 kl. 13:26 skrev Calle Hedberg calle.hedberg@gmail.com:

Hi,

In 2.28. the table “TrackedEntityAttributeReservedValues” is storing unique ID values that have been reserved. When a request is coming from e.g. an Android device running Tracker Capture, the API will trigger the reservation of 100 values and return these to the device while also storing the reserved values in the above table.

Yesterday, I inserted a list of such reserved values into that table in order to “protect” a section of numbers that will be used for importing legacy data.

I then decided to verify that those case IDs WERE now actually reserved.

when using the normal browser Tracker Capture, it clearly worked - no case IDs coming up were in the reserved range

But when I fired up Android Tracker Capture and logged in afresh, the system did reserve 100 more values BUT a number of those were values already in the TrackedEntityAttributeReservedValues table.

Or in other words, it seems like a request for reserved values from Tracker Capture via the API will generate reserved values without checking if those values are reserved already - is that correct?

Regards

Calle


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg



Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help : https://help.launchpad.net/ListHelp


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg



Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg