How do i debug an enrollment form, to track an issue on the DHIS2 mobile app

samuel.awodire · 1 December 2021 08:30

I have an issue with a particular program’s enrollment form, when i try to load the form on my dhis2 mobile app.
I get this log error on android studio, and no form is generated on the mobile app.

2021-12-01 09:18:29.614 19808-19973/org.e4e.dhis2.android W/System.err: 2021-12-01 09:18:29.615 19808-19973/org.e4e.dhis2.android W/System.err: 2021-12-01 09:18:29.615 19808-19973/org.e4e.dhis2.android W/System.err: 2021-12-01 09:18:29.615 19808-19973/org.e4e.dhis2.android W/System.err: 2021-12-01 09:18:29.615 19808-19973/org.e4e.dhis2.android W/System.err: 2021-12-01 09:18:29.615 19808-19973/org.e4e.dhis2.android W/System.err: 2021-12-01 09:18:29.615 19808-19973/org.e4e.dhis2.android W/System.err: 2021-12-01 09:18:29.615 19808-19973/org.e4e.dhis2.android W/System.err: 2021-12-01 09:18:29.615 19808-19973/org.e4e.dhis2.android W/System.err: 2021-12-01 09:18:29.615 19808-19973/org.e4e.dhis2.android W/System.err: 2021-12-01 09:18:29.615 19808-19973/org.e4e.dhis2.android W/System.err: 2021-12-01 09:18:29.615 19808-19973/org.e4e.dhis2.android W/System.err: 2021-12-01 09:18:29.615 19808-19973/org.e4e.dhis2.android W/System.err: 2021-12-01 09:18:29.615 19808-19973/org.e4e.dhis2.android W/System.err: 2021-12-01 09:18:29.615 19808-19973/org.e4e.dhis2.android W/System.err: 2021-12-01 09:18:29.615 19808-19973/org.e4e.dhis2.android W/System.err: 2021-12-01 09:18:29.615 19808-19973/org.e4e.dhis2.android W/System.err: 2021-12-01 09:18:29.615 19808-19973/org.e4e.dhis2.android W/System.err: 2021-12-01 09:18:29.615 19808-19973/org.e4e.dhis2.android W/System.err: 2021-12-01 09:18:29.615 19808-19973/org.e4e.dhis2.android W/System.err: 2021-12-01 09:18:29.615 19808-19973/org.e4e.dhis2.android W/System.err: 2021-12-01 09:18:29.615 19808-19973/org.e4e.dhis2.android W/System.err: 2021-12-01 09:18:29.615 19808-19973/org.e4e.dhis2.android W/System.err: 2021-12-01 09:18:29.615 19808-19973/org.e4e.dhis2.android W/System.err: 2021-12-01 09:18:29.615 19808-19973/org.e4e.dhis2.android W/System.err: 2021-12-01 09:18:29.616 19808-19973/org.e4e.dhis2.android W/System.err: 2021-12-01 09:18:29.616 19808-19973/org.e4e.dhis2.android W/System.err: 2021-12-01 09:18:29.616 19808-19973/org.e4e.dhis2.android W/System.err: 2021-12-01 09:18:29.616 19808-19973/org.e4e.dhis2.android W/System.err: 2021-12-01 09:18:29.616 19808-19973/org.e4e.dhis2.android W/System.err: 2021-12-01 09:18:29.616 19808-19973/org.e4e.dhis2.android W/System.err: 2021-12-01 09:18:29.616 19808-19973/org.e4e.dhis2.android W/System.err: 2021-12-01 09:18:29.616 19808-19973/org.e4e.dhis2.android W/System.err: 2021-12-01 09:18:29.616 19808-19973/org.e4e.dhis2.android W/System.err: 2021-12-01 09:18:29.616 19808-19973/org.e4e.dhis2.android W/System.err: 2021-12-01 09:18:29.616 19808-19973/org.e4e.dhis2.android W/System.err: 2021-12-01 09:18:29.616 19808-19973/org.e4e.dhis2.android W/System.err: org.hisp.dhis.antlr.ParserException: Invalid string token ‘or’ at line:1 character:65
at org.hisp.dhis.antlr.ParserErrorListener.syntaxError(ParserErrorListener.java:65)
at org.antlr.v4.runtime.ProxyErrorListener.syntaxError(ProxyErrorListener.java:41)
at org.antlr.v4.runtime.Parser.notifyErrorListeners(Parser.java:544)
at org.antlr.v4.runtime.DefaultErrorStrategy.reportMissingToken(DefaultErrorStrategy.java:409)
at org.antlr.v4.runtime.DefaultErrorStrategy.singleTokenInsertion(DefaultErrorStrategy.java:519)
at org.antlr.v4.runtime.DefaultErrorStrategy.recoverInline(DefaultErrorStrategy.java:476)
at org.antlr.v4.runtime.Parser.match(Parser.java:206)
at org.hisp.dhis.parser.expression.antlr.ExpressionParser.expr(ExpressionParser.java:2767)
at org.hisp.dhis.parser.expression.antlr.ExpressionParser.expr(ExpressionParser.java:408)
at org.hisp.dhis.parser.expression.antlr.ExpressionParser.expr(ExpressionParser.java:3076)
at org.hisp.dhis.parser.expression.antlr.ExpressionParser.expression(ExpressionParser.java:200)
at org.hisp.dhis.antlr.Parser.createParseTree(Parser.java:176)
at org.hisp.dhis.antlr.Parser.access$000(Parser.java:52)
at org.hisp.dhis.antlr.Parser$1.getValue(Parser.java:147)
at org.hisp.dhis.antlr.Parser$1.getValue(Parser.java:143)
at org.hisp.dhis.antlr.cache.LocalCache.get(LocalCache.java:103)
at org.hisp.dhis.antlr.Parser.getParseTree(Parser.java:142)
at org.hisp.dhis.antlr.Parser.visit(Parser.java:83)
at org.hisp.dhis.rules.RuleEngineExecution.process(RuleEngineExecution.java:117)
at org.hisp.dhis.rules.RuleEngineExecution.call(RuleEngineExecution.java:77)
at org.hisp.dhis.rules.RuleEngineExecution.call(RuleEngineExecution.java:18)
at org.e4e.dhis2.android.ui.enrollment_form.EnrollmentFormActivity.lambda$onResume$4$EnrollmentFormActivity(EnrollmentFormActivity.java:237)
at org.e4e.dhis2.android.ui.enrollment_form.EnrollmentFormActivity$$ExternalSyntheticLambda11.call(Unknown Source:4)
at io.reactivex.internal.operators.flowable.FlowableFromCallable.call(FlowableFromCallable.java:55)
at io.reactivex.internal.operators.flowable.FlowableScalarXMap.tryScalarXMapSubscribe(FlowableScalarXMap.java:80)
at io.reactivex.internal.operators.flowable.FlowableFlatMap.subscribeActual(FlowableFlatMap.java:50)
at io.reactivex.Flowable.subscribe(Flowable.java:14826)
at io.reactivex.Flowable.subscribe(Flowable.java:14773)
at io.reactivex.internal.operators.flowable.FlowableSubscribeOn$SubscribeOnSubscriber.run(FlowableSubscribeOn.java:82)
at io.reactivex.internal.schedulers.ScheduledRunnable.run(ScheduledRunnable.java:66)
at io.reactivex.internal.schedulers.ScheduledRunnable.call(ScheduledRunnable.java:57)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:301)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
at java.lang.Thread.run(Thread.java:919)

FYI, I am using the Dhis2 skeleton app codebase from github as a template, and other enrollment and event forms are generated for different programs successfully without any issue.
@vgarciabnz Thanks, for all your help so far, I really appreciate it

Daler · 1 December 2021 09:53

Check for ‘or’ string as this error says. Also check this place (EnrollmentFormActivity.java:237). Whats there and what do u pass

vgarciabnz · 1 December 2021 11:47

Hi @samuel.awodire,
could you share the program rule expression to check it? It looks like it contains a ‘or’ string; if this ‘or’ string acts as a logical operator, I would try to replace it by ‘||’ although it should be supported.

samuel.awodire · 2 December 2021 09:50

This is the line of code, throwing that exception
@Daler @vgarciabnz

        disposable.add(
                engineInitialization
                        .flatMap(next ->
                                Flowable.zip(
                                        EnrollmentFormService.getInstance().getEnrollmentFormFields()
                                                .subscribeOn(Schedulers.io()),
                                        engineService.ruleEnrollment().flatMap(ruleEnrollment ->
                                                Flowable.fromCallable(() -> ruleEngine.evaluate(ruleEnrollment).call()))
                                                .subscribeOn(Schedulers.io()),
                                        this::applyEffects
                                ))
                        .subscribeOn(Schedulers.io())
                        .observeOn(AndroidSchedulers.mainThread())
                        .subscribe(
                                fieldData -> {
                                    adapter.updateData(fieldData);
                                    finishedLoading();
                                },
                                Throwable::printStackTrace
                        )
        );

vgarciabnz · 2 December 2021 10:37

Can you share the expression of the program rule causing the issue? Or all the program rules if you are not sure about the one causing the issue. I think the first thing to do is to identify any part of the expression that could cause it.

samuel.awodire · 2 December 2021 11:07

@vgarciabnz
This is the ruleEnrollment() method

  public Flowable<RuleEnrollment> ruleEnrollment() {
        return Flowable.fromCallable(() -> {
            Enrollment enrollment = d2.enrollmentModule().enrollments().uid(enrollmentUid).blockingGet();
            String ouCode = d2.organisationUnitModule().organisationUnits().uid(enrollment.organisationUnit())
                    .blockingGet().code();
            Program program = d2.programModule().programs().uid(enrollment.program()).blockingGet();
            List<ProgramTrackedEntityAttribute> programTrackedEntityAttributes =
                    d2.programModule().programTrackedEntityAttributes()
                            .byProgram().eq(enrollment.program()).blockingGet();
            List<String> programAttributesUids = getProgramTrackedEntityAttributesUids(programTrackedEntityAttributes);

            List<RuleAttributeValue> attributeValues = transformToRuleAttributeValues(
                    d2.trackedEntityModule().trackedEntityAttributeValues()
                            .byTrackedEntityInstance().eq(enrollment.trackedEntityInstance())
                            .byTrackedEntityAttribute().in(programAttributesUids)
                            .blockingGet()
            );
            return RuleEnrollment.create(
                    enrollment.uid(),
                    enrollment.incidentDate(),
                    enrollment.enrollmentDate(),
                    enrollment.status() != null ?
                            RuleEnrollment.Status.valueOf(enrollment.status().name()) :
                            RuleEnrollment.Status.ACTIVE,
                    enrollment.organisationUnit(),
                    ouCode,
                    attributeValues,
                    program.name()
            );
        });
    }

@vgarciabnz This is the getEnrollmentFormFields() method


    public Flowable<Map<String, FormField>> getEnrollmentFormFields() {
        if (d2 == null)
            return Flowable.error(
                    new NullPointerException("D2 is null. EnrollmentForm has not been initialized, use init() function.")
            );
        else
            return Flowable.fromCallable(() ->
                    d2.programModule().programTrackedEntityAttributes()
                            .byProgram().eq(enrollmentRepository.blockingGet().program()).blockingGet()
            )
                    .flatMapIterable(programTrackedEntityAttributes -> programTrackedEntityAttributes)
                    .map(programAttribute -> {

                        TrackedEntityAttribute attribute = d2.trackedEntityModule().trackedEntityAttributes()
                                .uid(programAttribute.trackedEntityAttribute().uid())
                                .blockingGet();
                        TrackedEntityAttributeValueObjectRepository valueRepository =
                                d2.trackedEntityModule().trackedEntityAttributeValues()
                                        .value(programAttribute.trackedEntityAttribute().uid(),
                                                enrollmentRepository.blockingGet().trackedEntityInstance());

                        if (attribute.generated() && (valueRepository.blockingGet() == null || (valueRepository.blockingGet() != null &&
                                TextUtils.isEmpty(valueRepository.blockingGet().value())))) {
                            //get reserved value
                            String value = d2.trackedEntityModule().reservedValueManager()
                                    .blockingGetValue(programAttribute.trackedEntityAttribute().uid(),
                                            enrollmentRepository.blockingGet().organisationUnit());
                            valueRepository.blockingSet(value);
                        }

                        FormField field = new FormField(
                                attribute.uid(),
                                attribute.optionSet() != null ? attribute.optionSet().uid() : null,
                                attribute.valueType(),
                                String.format("%s%s", attribute.formName(), programAttribute.mandatory() ? "*" : ""),
                                valueRepository.blockingExists() ? valueRepository.blockingGet().value() : null,
                                null,
                                !attribute.generated(),
                                attribute.style()
                        );


                        fieldMap.put(programAttribute.trackedEntityAttribute().uid(), field);
                        return programAttribute;
                    }).toList().toFlowable()
                    .map(list -> fieldMap);
    }

samuel.awodire · 3 December 2021 06:15

@vgarciabnz Please, can you show me what an expression looks like from your end.

I think this will help me, find exactly what you’ve asked me to provide.

vgarciabnz · 3 December 2021 07:46

Sure, I meant the expression that defines the program rule. Using the Maintenance webapp, you can access the expression in the Program > Program rules:

If you want to extract all the expression at once, you can do that using the web api, something like this:
https://play.dhis2.org/2.36.4/api/programRules.json?fields=id,name,condition&paging=false. The purpose is just to identify any program rule that might be causing the issue.

samuel.awodire · 7 December 2021 15:19

@vgarciabnz This is all the programRule from the tracker, i am using for the Dhis2 mobile app.
I’m sorry it is coming very late, i had to contact my other team member to get it.

samuel.awodire · 9 December 2021 08:54

@vgarciabnz Please, do you have any update on my request

Daler · 9 December 2021 09:44

@samuel.awodire There are two ways as I can see.

First, put the brake point in the place where it throws an error and see if you can enter to Debug mode and stepBystep look for the changes in expression validator. What kind of values are being passed. This way you can see which expressions is wrong or affecting it.

Second option, remove all your program rules and try to load the program without any rules. Maybe an error is somewhere else. If it will load with any program rules, then you can slowly add back program rules and load the form until you will not find the one with issue.

There are always a slow process of debug/analysis which you should perform. its sometimes very difficult to identify issue remotely.

Hope it helps you to move forward or at least will get you to some idea .

samuel.awodire · 9 December 2021 10:24

@Daler Alright, i will try that. Thanks

vgarciabnz · 9 December 2021 11:48

Hi @samuel.awodire,

I could reproduce the issue locally and found out the problem. The program rule condition does not accept some reserved keywords, like “or”. In recent DHIS2 versions, the Maintenance app does not allow the creation of these rules and throws the error:

So, the solution is to rename the variables and avoid using “or” in the name. It would imply to refactor the program rule conditions as well.

samuel.awodire · 9 December 2021 12:56

@vgarciabnz Thanks so much for your help, i will pass this message directly to my team.
@Daler Thanks a lot, i really appreciate your help as well

samuel.awodire · 17 December 2021 09:38

@vgarciabnz I have been going back and forth with my team on this issue.
And, they said the program rule has a lot of “or” and “and”, so it is difficult to change.

Please, is there another work around i can do from my end, to just make the enrollment form work or that solution you preferred to me, is the only way.

@vgarciabnz your feedback will be appreciated, Thanks a lot

vgarciabnz · 17 December 2021 14:30

Hi @samuel.awodire,

I am afraid that those keywords are reserved and they will make the condition fail when is evaluated in the backend or android. It probably works in the Tracker capture webapp and this is why the issues has not been noticed before.

@Pablo, @Markus, @enrico, can you think of any workaround to avoid changing the name of the program rule variables?

enrico · 20 December 2021 07:28

Hi @samuel.awodire,
Unfortunately it is not possible to have Program Rule Variables named with token in them that are part of the grammar used in rule engine.
So something like “Apgar or score” is not valid and this validation was introduces in the api not too long ago so you may have been using a version where the validation is not present.
One workaround that we use is to create a SQL script to replace all occurrences of " and ", " or " and " not " with something different like " and ", " or " and " not ".
Sorry for the inconvenience but we really do not have a way to make the grammar to ignore keywords inside Program Rule Variable names

Gassim · 20 December 2021 09:01

@samuel.awodire I see that you basically have these to change (it might be possible to create an API script to change them so I hope it won’t be a difficult task for a developer):
#{B2. Babys condition at discharge or 7th day}
#{B4. Babys condition at discharge or 7th day new}
#{Baby condition at discharge or on the 7th day of life}
#{Baby condition at discharge or on the 7th day of life new}
#{Number of previous miscarriage or abortion DE}

jaime.bosque · 20 December 2021 09:45

Another option would be downloading the metadata of the program, editing with a text editor (using text replace) and reimporting the metadata. I think that should work and could be the easiest/quickest.

Gassim · 20 December 2021 09:47