I am trying to craft a web api call to determine if a dataset has at least one data value in it or not. I am not interested in the data values themselves, just if any data has been uploaded / inputted into the system yet or not.
If data values did exist, I would not know what org units or periods were chosen so would not be able to supply those parameters.
I would also prefer to do this in one api call instead of having to iterate through data elements until finding data (if any).
I’ve been digging around the api and the best I can find right now in one call is this:
but the “::isNotEmpty” does not seem to be in place for this call. I also could not seem to return paged data to reduce the payload.
Do you have any other suggestions of how I can do this? To give context, I will be iterating through all Datasets in the system and displaying them in a list, so I need something that is fairly efficient. I wouldn’t want to have to iterate through each data element individually unless necessary.
I think you have found the best approach to the problem.
I have added support in trunk now for a “limit” query parameter that puts a limit on the number of records to return. So you can now append “&limit=1” to your query and check if any records are returned.
On Mon, Jun 8, 2015 at 11:30 PM, Lorill Crees lcrees@2paths.com wrote:
Hi,
I am trying to craft a web api call to determine if a dataset has at least one data value in it or not. I am not interested in the data values themselves, just if any data has been uploaded / inputted into the system yet or not.
If data values did exist, I would not know what org units or periods were chosen so would not be able to supply those parameters.
I would also prefer to do this in one api call instead of having to iterate through data elements until finding data (if any).
I’ve been digging around the api and the best I can find right now in one call is this:
but the “::isNotEmpty” does not seem to be in place for this call. I also could not seem to return paged data to reduce the payload.
Do you have any other suggestions of how I can do this? To give context, I will be iterating through all Datasets in the system and displaying them in a list, so I need something that is fairly efficient. I wouldn’t want to have to iterate through each data element individually unless necessary.
On Tue, Jun 9, 2015 at 11:07 AM, Lars Helge Øverland larshelge@gmail.com wrote:
Hi Lorill,
I think you have found the best approach to the problem.
I have added support in trunk now for a “limit” query parameter that puts a limit on the number of records to return. So you can now append “&limit=1” to your query and check if any records are returned.
On Mon, Jun 8, 2015 at 11:30 PM, Lorill Crees lcrees@2paths.com wrote:
Hi,
I am trying to craft a web api call to determine if a dataset has at least one data value in it or not. I am not interested in the data values themselves, just if any data has been uploaded / inputted into the system yet or not.
If data values did exist, I would not know what org units or periods were chosen so would not be able to supply those parameters.
I would also prefer to do this in one api call instead of having to iterate through data elements until finding data (if any).
I’ve been digging around the api and the best I can find right now in one call is this:
but the “::isNotEmpty” does not seem to be in place for this call. I also could not seem to return paged data to reduce the payload.
Do you have any other suggestions of how I can do this? To give context, I will be iterating through all Datasets in the system and displaying them in a list, so I need something that is fairly efficient. I wouldn’t want to have to iterate through each data element individually unless necessary.
I wonder if you also have any suggestions for checking if data has been imported for Programs and ProgramStages.
For Programs, I have been doing the following:
/api/enrollments?program=
Get the orgUnit from the first enrollment
/api/trackedEntityInstances?ou=&program=
However, this has poor performance and a large number of records can be returned.
For ProgramStages, I have been doing the following:
/api/events?programStage=
Again, poor performance and a large number of records can be returned.
One thing I need to display a listing page with all Programs/ProgramStages and show whether there is data assigned. A faster method of querying this is essential.
Any suggestions for more performant queries would be greatly appreciated.
Many thanks
Alan
···
On Tue, Jun 9, 2015 at 3:18 AM, Lars Helge Øverland larshelge@gmail.com wrote:
On Tue, Jun 9, 2015 at 11:07 AM, Lars Helge Øverland larshelge@gmail.com wrote:
Hi Lorill,
I think you have found the best approach to the problem.
I have added support in trunk now for a “limit” query parameter that puts a limit on the number of records to return. So you can now append “&limit=1” to your query and check if any records are returned.
On Mon, Jun 8, 2015 at 11:30 PM, Lorill Crees lcrees@2paths.com wrote:
Hi,
I am trying to craft a web api call to determine if a dataset has at least one data value in it or not. I am not interested in the data values themselves, just if any data has been uploaded / inputted into the system yet or not.
If data values did exist, I would not know what org units or periods were chosen so would not be able to supply those parameters.
I would also prefer to do this in one api call instead of having to iterate through data elements until finding data (if any).
I’ve been digging around the api and the best I can find right now in one call is this:
but the “::isNotEmpty” does not seem to be in place for this call. I also could not seem to return paged data to reduce the payload.
Do you have any other suggestions of how I can do this? To give context, I will be iterating through all Datasets in the system and displaying them in a list, so I need something that is fairly efficient. I wouldn’t want to have to iterate through each data element individually unless necessary.
Thanks for this. I’ve tried it out and it’s working well in terms of functionality, but the performance is very poor. Even if I’m limiting to just one record, one api call is taking 5-7 seconds. When needing to iterate over 17 datasets this means two minutes for my page load which would not be acceptable.
Are there any performance improvements that can be made for this?
Let me know. Thanks,
Lorill
···
On Tue, Jun 9, 2015 at 3:18 AM, Lars Helge Øverland larshelge@gmail.com wrote:
This has been backported to 2.19 now.
On Tue, Jun 9, 2015 at 11:07 AM, Lars Helge Øverland larshelge@gmail.com wrote:
Hi Lorill,
I think you have found the best approach to the problem.
I have added support in trunk now for a “limit” query parameter that puts a limit on the number of records to return. So you can now append “&limit=1” to your query and check if any records are returned.
On Mon, Jun 8, 2015 at 11:30 PM, Lorill Crees lcrees@2paths.com wrote:
Hi,
I am trying to craft a web api call to determine if a dataset has at least one data value in it or not. I am not interested in the data values themselves, just if any data has been uploaded / inputted into the system yet or not.
If data values did exist, I would not know what org units or periods were chosen so would not be able to supply those parameters.
I would also prefer to do this in one api call instead of having to iterate through data elements until finding data (if any).
I’ve been digging around the api and the best I can find right now in one call is this:
but the “::isNotEmpty” does not seem to be in place for this call. I also could not seem to return paged data to reduce the payload.
Do you have any other suggestions of how I can do this? To give context, I will be iterating through all Datasets in the system and displaying them in a list, so I need something that is fairly efficient. I wouldn’t want to have to iterate through each data element individually unless necessary.
I have done an optimization now in trunk r 19326 and 2.19. It seems we unnecessarily loaded persisted periods for the time span when start and end dates were used. Please try again on latest trunk and see if it works better now.
Lars
···
On Tue, Jun 9, 2015 at 8:21 PM, Lorill Crees lcrees@2paths.com wrote:
Hi Lars,
Thanks for this. I’ve tried it out and it’s working well in terms of functionality, but the performance is very poor. Even if I’m limiting to just one record, one api call is taking 5-7 seconds. When needing to iterate over 17 datasets this means two minutes for my page load which would not be acceptable.
Are there any performance improvements that can be made for this?
Let me know. Thanks,
Lorill
–
On Tue, Jun 9, 2015 at 3:18 AM, Lars Helge Øverland larshelge@gmail.com wrote:
This has been backported to 2.19 now.
On Tue, Jun 9, 2015 at 11:07 AM, Lars Helge Øverland larshelge@gmail.com wrote:
Hi Lorill,
I think you have found the best approach to the problem.
I have added support in trunk now for a “limit” query parameter that puts a limit on the number of records to return. So you can now append “&limit=1” to your query and check if any records are returned.
On Mon, Jun 8, 2015 at 11:30 PM, Lorill Crees lcrees@2paths.com wrote:
Hi,
I am trying to craft a web api call to determine if a dataset has at least one data value in it or not. I am not interested in the data values themselves, just if any data has been uploaded / inputted into the system yet or not.
If data values did exist, I would not know what org units or periods were chosen so would not be able to supply those parameters.
I would also prefer to do this in one api call instead of having to iterate through data elements until finding data (if any).
I’ve been digging around the api and the best I can find right now in one call is this:
but the “::isNotEmpty” does not seem to be in place for this call. I also could not seem to return paged data to reduce the payload.
Do you have any other suggestions of how I can do this? To give context, I will be iterating through all Datasets in the system and displaying them in a list, so I need something that is fairly efficient. I wouldn’t want to have to iterate through each data element individually unless necessary.