We are working on DHIS2 TB-10,11 and 12 forms (for Bangladesh) to integrate with external eTB manager application . The TB forms have developed in DHIS2 for collecting TB -10,11 and 12 data from Upazila level in Bangladesh but we have a local level application(eTBM) that is using to collect individual case record and generate aggregated TB-10,11 and 12 reports. Now our goal is to integrate eTMB with DHIS2 TB forms.
Already we have tested the integration that is working fine. But we are facing a problem with deleting existing records of these forms. I have a question on it:
Is there any way of requesting DHIS2 to clear or delete the whole TB 10 form for a specific orgunit and period before sending the external data?
Thank you very much for your co-operations.
···
If you have any questions or concerns, don’t hesitate to let me know.
Here in Rwanda, we have implemented TB and MDR TB tracking systems in DHIS2, collecting summarized patient level data to generate quarterly forms but when it comes to delete records for different reasons into this multi-stages system we re also having hard time to delete recorded data. dhis2.25 has an option of deleting cases reported and through Data administration-Maintenance you can “Permanently remove soft deleted data values” , for some case it works and if i am not mistaken you delete records one by one, you can’t select a list of records and delete them once
We are working on DHIS2 TB-10,11 and 12 forms (for Bangladesh) to integrate with external eTB manager application . The TB forms have developed in DHIS2 for collecting TB -10,11 and 12 data from Upazila level in Bangladesh but we have a local level application(eTBM) that is using to collect individual case record and generate aggregated TB-10,11 and 12 reports. Now our goal is to integrate eTMB with DHIS2 TB forms.
Already we have tested the integration that is working fine. But we are facing a problem with deleting existing records of these forms. I have a question on it:
Is there any way of requesting DHIS2 to clear or delete the whole TB 10 form for a specific orgunit and period before sending the external data?
Thank you very much for your co-operations.
If you have any questions or concerns, don’t hesitate to let me know.
Thank you very much. Yes we are deleting records one by one but we need to delete all records of a form for a single orgunit and period.
···
On Mon, Feb 27, 2017 at 8:14 PM, Kamugunga Adolphe kaadol@gmail.com wrote:
Dear Sujan,
Here in Rwanda, we have implemented TB and MDR TB tracking systems in DHIS2, collecting summarized patient level data to generate quarterly forms but when it comes to delete records for different reasons into this multi-stages system we re also having hard time to delete recorded data. dhis2.25 has an option of deleting cases reported and through Data administration-Maintenance you can “Permanently remove soft deleted data values” , for some case it works and if i am not mistaken you delete records one by one, you can’t select a list of records and delete them once
We are working on DHIS2 TB-10,11 and 12 forms (for Bangladesh) to integrate with external eTB manager application . The TB forms have developed in DHIS2 for collecting TB -10,11 and 12 data from Upazila level in Bangladesh but we have a local level application(eTBM) that is using to collect individual case record and generate aggregated TB-10,11 and 12 reports. Now our goal is to integrate eTMB with DHIS2 TB forms.
Already we have tested the integration that is working fine. But we are facing a problem with deleting existing records of these forms. I have a question on it:
Is there any way of requesting DHIS2 to clear or delete the whole TB 10 form for a specific orgunit and period before sending the external data?
Thank you very much for your co-operations.
If you have any questions or concerns, don’t hesitate to let me know.
that’s an interesting problem. We could certainly implement support for deletion per org unit/period.
I would however recommend another approach: For each aggregate report you send from eTBM, make sure you send a value, including 0, for each field in the aggregate TB form. This will make your information more explicit, ie. you communicate that for all data element + category option combos exactly how many cases you have. The benefit then is that if cases are deleted in the upstream system (eTBM), it will be correctly reflected in the aggregate form in DHIS 2.
This will be semantically more correct than simply dropping all data values first. I.e. when you look at the audit logs for each data element, you will see that a value changed from a positive value, then potentially to a 0 value. This will look better compared to audit logs showing that all values were deleted every day. This will improve auditability and data validation.
Of course this might lead to the aggregate export payload becoming a bit bigger, but I think that can be justified. You can also make sure that the TB data elements are set to “zero not significant”, which will delete or not store 0 values on the DHIS 2 side.
We are working on DHIS2 TB-10,11 and 12 forms (for Bangladesh) to integrate with external eTB manager application . The TB forms have developed in DHIS2 for collecting TB -10,11 and 12 data from Upazila level in Bangladesh but we have a local level application(eTBM) that is using to collect individual case record and generate aggregated TB-10,11 and 12 reports. Now our goal is to integrate eTMB with DHIS2 TB forms.
Already we have tested the integration that is working fine. But we are facing a problem with deleting existing records of these forms. I have a question on it:
Is there any way of requesting DHIS2 to clear or delete the whole TB 10 form for a specific orgunit and period before sending the external data?
Thank you very much for your co-operations.
If you have any questions or concerns, don’t hesitate to let me know.
Thank you very much. Yes this is possible to set zero for each field. I will follow your recommendation and share with you the update.
···
On Mon, Feb 27, 2017 at 8:59 PM, Lars Helge Øverland lars@dhis2.org wrote:
Hi Julhas,
that’s an interesting problem. We could certainly implement support for deletion per org unit/period.
I would however recommend another approach: For each aggregate report you send from eTBM, make sure you send a value, including 0, for each field in the aggregate TB form. This will make your information more explicit, ie. you communicate that for all data element + category option combos exactly how many cases you have. The benefit then is that if cases are deleted in the upstream system (eTBM), it will be correctly reflected in the aggregate form in DHIS 2.
This will be semantically more correct than simply dropping all data values first. I.e. when you look at the audit logs for each data element, you will see that a value changed from a positive value, then potentially to a 0 value. This will look better compared to audit logs showing that all values were deleted every day. This will improve auditability and data validation.
Of course this might lead to the aggregate export payload becoming a bit bigger, but I think that can be justified. You can also make sure that the TB data elements are set to “zero not significant”, which will delete or not store 0 values on the DHIS 2 side.
We are working on DHIS2 TB-10,11 and 12 forms (for Bangladesh) to integrate with external eTB manager application . The TB forms have developed in DHIS2 for collecting TB -10,11 and 12 data from Upazila level in Bangladesh but we have a local level application(eTBM) that is using to collect individual case record and generate aggregated TB-10,11 and 12 reports. Now our goal is to integrate eTMB with DHIS2 TB forms.
Already we have tested the integration that is working fine. But we are facing a problem with deleting existing records of these forms. I have a question on it:
Is there any way of requesting DHIS2 to clear or delete the whole TB 10 form for a specific orgunit and period before sending the external data?
Thank you very much for your co-operations.
If you have any questions or concerns, don’t hesitate to let me know.
Please be note that, for security reason we are not allowing ‘data delete’ from any data set through API externally if it is not exclusive (data only enter through API). So audit log is monitored. If identifies the user could be blocked. As your eTB system suppose to be send aggregated data to national country wide DHIS2, make sure that, it is tested properly, reviewed by us and only then you could have access to the live production system.
API from external source are only allowed ‘insert’ and ‘update’. System security will not be compromised.
Regards
···
On Mon, Feb 27, 2017 at 8:59 PM, Lars Helge Øverland lars@dhis2.org wrote:
Hi Julhas,
that’s an interesting problem. We could certainly implement support for deletion per org unit/period.
I would however recommend another approach: For each aggregate report you send from eTBM, make sure you send a value, including 0, for each field in the aggregate TB form. This will make your information more explicit, ie. you communicate that for all data element + category option combos exactly how many cases you have. The benefit then is that if cases are deleted in the upstream system (eTBM), it will be correctly reflected in the aggregate form in DHIS 2.
This will be semantically more correct than simply dropping all data values first. I.e. when you look at the audit logs for each data element, you will see that a value changed from a positive value, then potentially to a 0 value. This will look better compared to audit logs showing that all values were deleted every day. This will improve auditability and data validation.
Of course this might lead to the aggregate export payload becoming a bit bigger, but I think that can be justified. You can also make sure that the TB data elements are set to “zero not significant”, which will delete or not store 0 values on the DHIS 2 side.
We are working on DHIS2 TB-10,11 and 12 forms (for Bangladesh) to integrate with external eTB manager application . The TB forms have developed in DHIS2 for collecting TB -10,11 and 12 data from Upazila level in Bangladesh but we have a local level application(eTBM) that is using to collect individual case record and generate aggregated TB-10,11 and 12 reports. Now our goal is to integrate eTMB with DHIS2 TB forms.
Already we have tested the integration that is working fine. But we are facing a problem with deleting existing records of these forms. I have a question on it:
Is there any way of requesting DHIS2 to clear or delete the whole TB 10 form for a specific orgunit and period before sending the external data?
Thank you very much for your co-operations.
If you have any questions or concerns, don’t hesitate to let me know.
On Mon, Feb 27, 2017 at 8:59 PM, Lars Helge Øverland lars@dhis2.org wrote:
Hi Julhas,
that’s an interesting problem. We could certainly implement support for deletion per org unit/period.
I would however recommend another approach: For each aggregate report you send from eTBM, make sure you send a value, including 0, for each field in the aggregate TB form. This will make your information more explicit, ie. you communicate that for all data element + category option combos exactly how many cases you have. The benefit then is that if cases are deleted in the upstream system (eTBM), it will be correctly reflected in the aggregate form in DHIS 2.
This will be semantically more correct than simply dropping all data values first. I.e. when you look at the audit logs for each data element, you will see that a value changed from a positive value, then potentially to a 0 value. This will look better compared to audit logs showing that all values were deleted every day. This will improve auditability and data validation.
Of course this might lead to the aggregate export payload becoming a bit bigger, but I think that can be justified. You can also make sure that the TB data elements are set to “zero not significant”, which will delete or not store 0 values on the DHIS 2 side.
We are working on DHIS2 TB-10,11 and 12 forms (for Bangladesh) to integrate with external eTB manager application . The TB forms have developed in DHIS2 for collecting TB -10,11 and 12 data from Upazila level in Bangladesh but we have a local level application(eTBM) that is using to collect individual case record and generate aggregated TB-10,11 and 12 reports. Now our goal is to integrate eTMB with DHIS2 TB forms.
Already we have tested the integration that is working fine. But we are facing a problem with deleting existing records of these forms. I have a question on it:
Is there any way of requesting DHIS2 to clear or delete the whole TB 10 form for a specific orgunit and period before sending the external data?
Thank you very much for your co-operations.
If you have any questions or concerns, don’t hesitate to let me know.
Our plan is to implement DHIS2 tracker as well. We are gradually moving towards DHIS2 TB Tracker, so your experience will be helpful for us. Hopefully in the next financial year we can move towards DHIS2 TB tracker.
Regards
Hannan
···
On Mon, Feb 27, 2017 at 8:14 PM, Kamugunga Adolphe kaadol@gmail.com wrote:
Dear Sujan,
Here in Rwanda, we have implemented TB and MDR TB tracking systems in DHIS2, collecting summarized patient level data to generate quarterly forms but when it comes to delete records for different reasons into this multi-stages system we re also having hard time to delete recorded data. dhis2.25 has an option of deleting cases reported and through Data administration-Maintenance you can “Permanently remove soft deleted data values” , for some case it works and if i am not mistaken you delete records one by one, you can’t select a list of records and delete them once
We are working on DHIS2 TB-10,11 and 12 forms (for Bangladesh) to integrate with external eTB manager application . The TB forms have developed in DHIS2 for collecting TB -10,11 and 12 data from Upazila level in Bangladesh but we have a local level application(eTBM) that is using to collect individual case record and generate aggregated TB-10,11 and 12 reports. Now our goal is to integrate eTMB with DHIS2 TB forms.
Already we have tested the integration that is working fine. But we are facing a problem with deleting existing records of these forms. I have a question on it:
Is there any way of requesting DHIS2 to clear or delete the whole TB 10 form for a specific orgunit and period before sending the external data?
Thank you very much for your co-operations.
If you have any questions or concerns, don’t hesitate to let me know.
Thank you very much. Yes this is possible to set zero for each field. I will follow your recommendation and share with you the update.
Best Regards Julhas
On Mon, Feb 27, 2017 at 8:59 PM, Lars Helge Øverland lars@dhis2.org wrote:
Hi Julhas,
that’s an interesting problem. We could certainly implement support for deletion per org unit/period.
I would however recommend another approach: For each aggregate report you send from eTBM, make sure you send a value, including 0, for each field in the aggregate TB form. This will make your information more explicit, ie. you communicate that for all data element + category option combos exactly how many cases you have. The benefit then is that if cases are deleted in the upstream system (eTBM), it will be correctly reflected in the aggregate form in DHIS 2.
This will be semantically more correct than simply dropping all data values first. I.e. when you look at the audit logs for each data element, you will see that a value changed from a positive value, then potentially to a 0 value. This will look better compared to audit logs showing that all values were deleted every day. This will improve auditability and data validation.
Of course this might lead to the aggregate export payload becoming a bit bigger, but I think that can be justified. You can also make sure that the TB data elements are set to “zero not significant”, which will delete or not store 0 values on the DHIS 2 side.
We are working on DHIS2 TB-10,11 and 12 forms (for Bangladesh) to integrate with external eTB manager application . The TB forms have developed in DHIS2 for collecting TB -10,11 and 12 data from Upazila level in Bangladesh but we have a local level application(eTBM) that is using to collect individual case record and generate aggregated TB-10,11 and 12 reports. Now our goal is to integrate eTMB with DHIS2 TB forms.
Already we have tested the integration that is working fine. But we are facing a problem with deleting existing records of these forms. I have a question on it:
Is there any way of requesting DHIS2 to clear or delete the whole TB 10 form for a specific orgunit and period before sending the external data?
Thank you very much for your co-operations.
If you have any questions or concerns, don’t hesitate to let me know.
we are doing something like that (on the aggregate level) in a datawarehouse setting without 0-values, where we have to update whole datasets every night.
Basically we download all data with kettle, compare the differences to the incoming data, and then push only changed values back to DHIS2. Those values (category/attribute combinations) which are no longer needed, we push as delete requests.
That sounds very complicated (and might need a whole lot of memory on the etl side depending on the number of values you are dealing with) but has the advantage that you reduce the number of records to be pushed into DHIS2 to those that have really changed (and you keep the audit-logs as Lars suggested)
Hope that helps,
Uwe
···
Hannan Khan <hannank@gmail.com> hat am 27. Februar 2017 um 18:56 geschrieben:
We are not allowing '0' in the data until and unless it is essential. In
your case '0' will be you significant amount; so we cant allow that.
On Mon, Feb 27, 2017 at 9:25 PM, Julhas Sujan <julhaspustcse@gmail.com> > wrote:
> Hi Lars,
>
> Thank you very much. Yes this is possible to set zero for each field. I
> will follow your recommendation and share with you the update.
>
>
> *Best Regards*
>
> *Julhas *
>
> On Mon, Feb 27, 2017 at 8:59 PM, Lars Helge Øverland <lars@dhis2.org> > > wrote:
>
>> Hi Julhas,
>>
>> that's an interesting problem. We could certainly implement support for
>> deletion per org unit/period.
>>
>> I would however recommend another approach: For each aggregate report you
>> send from eTBM, make sure you send a value, including 0, for* each field*
>> in the aggregate TB form. This will make your information more explicit,
>> ie. you communicate that for all data element + category option combos
>> exactly how many cases you have. The benefit then is that if cases are
>> deleted in the upstream system (eTBM), it will be correctly reflected in
>> the aggregate form in DHIS 2.
>>
>> This will be semantically more correct than simply dropping all data
>> values first. I.e. when you look at the audit logs for each data element,
>> you will see that a value changed from a positive value, then potentially
>> to a 0 value. This will look better compared to audit logs showing that all
>> values were deleted every day. This will improve auditability and data
>> validation.
>>
>> Of course this might lead to the aggregate export payload becoming a bit
>> bigger, but I think that can be justified. You can also make sure that the
>> TB data elements are set to "zero not significant", which will delete or
>> not store 0 values on the DHIS 2 side.
>>
>> Let me know if this sounds reasonable.
>>
>> best,
>>
>> Lars
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> On Mon, Feb 27, 2017 at 12:43 PM, Julhas Sujan <julhaspustcse@gmail.com> > >> wrote:
>>
>>> Dear All,
>>>
>>> Good evening!
>>>
>>> We are working on DHIS2 TB-10,11 and 12 forms (for Bangladesh) to
>>> integrate with external eTB manager application
>>> <http://etbmanagerbd.org/>\. The TB forms have developed in DHIS2 for
>>> collecting TB -10,11 and 12 data from Upazila level in Bangladesh but we
>>> have a local level application(eTBM) that is using to collect individual
>>> case record and generate aggregated TB-10,11 and 12 reports. Now our goal
>>> is to integrate eTMB with DHIS2 TB forms.
>>>
>>> Already we have tested the integration that is working fine. But we are
>>> facing a problem with deleting existing records of these forms. I have a
>>> question on it:
>>>
>>> *Is there any way of requesting DHIS2 to clear or delete the whole TB 10
>>> form for a specific orgunit and period before sending the external data?*
>>>
>>>
>>> Thank you very much for your co-operations.
>>>
>>>
>>> *If you have any questions or concerns, don’t hesitate to let me know.*
>>>
>>> *Best Regards*
>>> *Julhas Sujan*
>>> Senior Java Programmer
>>> Management Science for Health(MSH)
>>> Dhaka, Bangladesh
>>> Cell: +8801989 442856
>>> Skype: julhas08
>>>
>>> www.msh.orghttp://siapsprogram.org/wherewework/bangladesh/
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~dhis2-users
>>> Post to : dhis2-users@lists.launchpad.net
>>> Unsubscribe : https://launchpad.net/~dhis2-users
>>> More help : https://help.launchpad.net/ListHelp
>>>
>>>
>>
>>
>> --
>> Lars Helge Øverland
>> Lead developer, DHIS 2
>> University of Oslo
>> Skype: larshelgeoverland
>> lars@dhis2.org
>> http://www.dhis2.org <https://www.dhis2.org/>
>>
>>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~dhis2-users
> Post to : dhis2-users@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~dhis2-users
> More help : https://help.launchpad.net/ListHelp
>
>
--
Muhammad Abdul Hannan Khan
DHIS2 Country coordinator & Secretary
HISP Bangladesh
