Prometheus and monitoring.api.enabled

so I’ve added monitoring option in dhis.conf

monitoring.api.enabled       = on

I see the logs it’s enabled

[localhost-startStop-1] INFO org.hisp.dhis.monitoring.metrics.MetricsEnabler - Monitoring metric for key monitoring.api.enabled is enabled
[localhost-startStop-1] INFO org.hisp.dhis.monitoring.metrics.MetricsEnabler - Monitoring metric for key monitoring.cpu.enabled is enabled
[localhost-startStop-1] INFO org.hisp.dhis.monitoring.metrics.MetricsEnabler - Monitoring metric for key monitoring.hibernate.enabled is disabled
[localhost-startStop-1] INFO org.hisp.dhis.monitoring.metrics.MetricsEnabler - Monitoring metric for key monitoring.uptime.enabled is enabled
[localhost-startStop-1] INFO org.hisp.dhis.monitoring.metrics.MetricsEnabler - Monitoring metric for key monitoring.dbpool.enabled is enabled
[localhost-startStop-1] INFO org.hisp.dhis.monitoring.metrics.MetricsEnabler - Monitoring metric for key monitoring.jvm.enabled is enabled

then after a dhis2 restart I access the dashboard, the api (/api/organisationUnits)
but the /api/metrics don’t return the api metrics
I only see the jvm and jdbc metrics

# HELP jdbc_connections_active  
# TYPE jdbc_connections_active gauge
jdbc_connections_active{name="actual",} 2.0
jdbc_connections_active{name="dataSource",} 2.0
# HELP system_cpu_usage The "recent cpu usage" for the whole system
# TYPE system_cpu_usage gauge
system_cpu_usage 0.03773584905660377
# HELP jvm_buffer_count_buffers An estimate of the number of buffers in the pool
# TYPE jvm_buffer_count_buffers gauge
jvm_buffer_count_buffers{id="direct",} 28.0
jvm_buffer_count_buffers{id="mapped",} 0.0
...

What am I missing to get the more api metric like showed in the doc

dhis2_seconds_max{exception="None",method="GET",outcome="SUCCESS",status="200",uri="/29/analytics",} 2.715992829
dhis2_seconds_sum{exception="None",method="GET",outcome="SUCCESS",status="200",uri="/29/analytics",} 15.404253163
dhis2_seconds_count{exception="None",method="GET",outcome="SUCCESS",status="200",uri="/29/analytics",} 40.0

just in case the version

  "version": "2.36.8",
  "revision": "aa38c7f",
  "buildTime": "2022-02-02T14:58:22.000",
3 Likes

ok the code I looked was WebMvcMetricsDisabledCondition vs RedisEnabledCondition

but it’s still don’t explain why I don’t have the api metrics details

I cannot recall exactly but I think there was an issue with the documentation, where you have to use true instead of on. Try that and see if it resolves the problem.

We have made a fix in recent versions to support both true and on for boolean properties.

best regards,

Lars

1 Like

Tried with true, nothing shows up for the api monitoring

# Enable monitoring
monitoring.api.enabled=true
monitoring.jvm.enabled=on

I’m not sure if it could be an actual problem but since it happens many times else where, would you try to use the incognito and check the /api/metrics again? If you see the results then clear your browser cache!

(even via curl, so without a browser)
/api/metrics returns the other metrics, but not the api one

1 Like

Thanks! Maybe the best I can do at this point is ask you for more specific info, please, so that I can triage for support again. Would you share the Prometheus configuration (without the sensitive info)? I see in the docs it also has a note:

Are you using this on Docker? Is it on Debian or another OS? If there’s any additional info that you think could help debug/diagnose the issue then please share!

Thank you!

The problem is on the dhis2 side (or my config). I have the /api/metrics return some metrics but not exposing the api metrics.
The dhis2 run in a docker image and an embedded tomcat but behind an aws load balancer.
The metrics arrive well in prometheus (except the api metrics).

If someone could at least confirm that in a production (or at least development) that they have the api measurement showing up in the /api/metrics and give the version and dhis.conf they used.

Is there some logging package I can put to debug (eg the metrics framework used ) ?

1 Like

@Stephan_Mestach , just wondering if you were able to solve this issue? Thanks!