This should now be fixed in latest 2.25, 2.24 and 2.23 (currently building on CI, war-file available within the hour).
Anyone using Tracker with File data values will need to upgrade. Please let me know if you have any more issues.
···
On Mon, Jul 25, 2016 at 5:50 PM, Halvdan Hoem Grelland halvdan@dhis2.org wrote:
To update everyone: this is definitely a bug with this stuff when using FileResource with Tracker (2.23 → ). I’m currently working on a fix, but it will take some time.
It hopefully won’t take long, but as it’s a pretty critical issue to have in production you will need to work around it in the mean time. The least intrusive way is to run a cronjob at just before the cleanup task triggers (2:00 server time) and update all fileresource rows in the DB with isassigned=TRUE. Alternately a patch could be applied (comments out a single line to disable the deletion/cleanup task). Let me know if you need any more details.
–
On Mon, Jul 25, 2016 at 2:06 PM, Halvdan Hoem Grelland halvdan@dhis2.org wrote:
Ok.
That task should detect unreferenced fileResource objects and delete them. If they are referenced, however (i.e. there is a data value referencing their UID) they should not be deleted. If the uid in your example is in fact stored in a data value with a FR data element there is a bug, if not it works as intended (to prevent orphaned files).
Could you check if the ‘assigned’ field of the FR is true after creating a new event with a FR data elements? If not, there is a problem in the server side event creation logic.
There is no way to disable the task (also, you really shouldn’t).
–
Halvdan Hoem Grelland
Software developer, DHIS 2
University of Oslo
http://www.dhis2.org
On Mon, Jul 25, 2016 at 1:37 PM, Mike Nelushi mikeevolution@gmail.com wrote:
Hi Halvdan,
We want the user to upload the file and be able to retrieve it but the files are deleted(purged) at 2AM everyday.
- INFO 2016-07-25 02:00:00,072 ‘system-process’ delete org.hisp.dhis.fileresource.FileResource, name: PERSONAL_CONSIDERATION_SUBMISSION. Samuel Mur 7910025121081.pdf, uid: ko0TqodGcPh (AuditLogUtil.java [taskScheduler-4])
- INFO 2016-07-25 02:00:00,082 ‘system-process’ delete org.hisp.dhis.fileresource.FileResource, name: Craig Anders Personal Consideration.pdf, uid: ShiqhAsXc78 (AuditLogUtil.java [taskScheduler-4])
- INFO 2016-07-25 02:00:00,096 ‘system-process’ delete org.hisp.dhis.fileresource.FileResource, name: Motivation for Comm - Copy.pdf, uid: L1fLHOSqwkM (AuditLogUtil.java [taskScheduler-4])
- INFO 2016-07-25 02:00:00,102 ‘system-process’ delete org.hisp.dhis.fileresource.FileResource, name: SATA - TOUCH WORLD CUP 2015 - SA MIXED TEAM CONFIRMATION.pdf, uid: EWQXKQsCunv (AuditLogUtil.java [taskScheduler-4])
- INFO 2016-07-25 02:00:00,108 ‘system-process’ delete org.hisp.dhis.fileresource.FileResource, name: N_Isaacs_WPTA_July2016.pdf, uid: AmoMiCsoSvD (AuditLogUtil.java [taskScheduler-4])
…
How do we disable this task from running?
Regards,
Mike
–
Halvdan Hoem Grelland
Software developer, DHIS 2
University of Oslo
http://www.dhis2.org
On Mon, Jul 25, 2016 at 1:27 PM, Halvdan Hoem Grelland halvdan@dhis2.org wrote:
This should be fine. Are you experiencing that it doesn’t work as expected? (I didn’t consider the tracker use case when I wrote ‘dataset’. My bad…)
Calle: your assumptions are correct. DV to FR is one-to-one. You would need a DE per specific file for your event/dataset, or alternately package multiple files in some archive format and store that instead (you could handle this on the client side).
What if you want multiple data value records to reference the same document […]
A FR data value is just a data value with a weak reference to a file. Grouping data elements is no different than for straight numeric or text values. That is, there is no ‘link’ between the fields, but you could use a construct such as a data set (aggr) or programstageinstance/event (tracker) to group DEs.
On Mon, Jul 25, 2016 at 1:02 PM, Mike Nelushi mikeevolution@gmail.com wrote:
Hi Halvdan,
The following are the steps I’m using:
- Save the file on the api/fileResource endpoint, get the UID back, then save the uid as of a dataValue on the event. Example:
POST to api/fileResources, get id: “a11111bbbb” back.
2. PUT the event:
{ "event" : "<some-event-uid>", "dataValues" : [ { "value" : "a11111bbbb", "dataElement" : "<some-fileResource-dataelement-uid>" } ] }
Are these steps correct?
Are the datasets required in these case? If yes, how should I go about doing this in the tracker module.
Regards,
Mike
–
Halvdan Hoem Grelland
Software developer, DHIS 2
University of Oslo
http://www.dhis2.org
On Mon, Jul 25, 2016 at 11:53 AM, Halvdan Hoem Grelland halvdan@dhis2.org wrote:
Hi Mike,
As is explained in the docs, the FR must be referenced by a data value. If the FR is referenced it will not be deleted.
The FR object is not usable by itself, so the upload to the api/*/fileResources is step 1 of 2 in the process of uploading a file (it needs to be tied to a DV afterwards).
In practice this means that in order to use FR you need to create data elements (and datasets) to contain your files.
Hope that answers your question. If not, please let me know.
On Mon, Jul 25, 2016 at 7:57 AM, Mike Nelushi mikeevolution@gmail.com wrote:
Hi Halvdan/Morten,
There seem to be a cleanup task which is ran at 2AM every 24H which necessary maintenance which removes orphaned files (FileResources) which have been posted to the instance but are marked for deletion as Halvdan explained here. And from File resource constraints explained **here “Any file resources which are in this state and are older than two hours will be marked for deletion and will eventually be purged from the system.” **
Our new application allows users to upload files and would like to prevent this from happening.
How do we stop the system from running this file deletion task or tag the files to not be deleted?
Please advice.
Regards,
Mike
–
Halvdan Hoem Grelland
Software developer, DHIS 2
University of Oslo
http://www.dhis2.org
Halvdan Hoem Grelland
Software developer, DHIS 2
University of Oslo
http://www.dhis2.org