Hi @Scott @Gassim do you know of a team who is dealing with a heavy Tracker program, and also using PowerBi? We are investigating performance slowness, and have a few things to deal with - too many Data Elements = massive analytics table (takes 10 minutes to push into PowerBi). I’ve contacted BAO to speak more on their integration suite, although I think it’s for aggregate…any one we might connect with for some troubleshooting advice? We have run into 1600 DE limit, which appears to be a system-wide limit for DHIS2 - and this is only one program. The program is a worldwide charity’s M&E tool, so there’s lots being collected. Thanks in advance!
Hi @sgaudon, Sorry I’m not really sure of anyone else doing those kinds of pushes to PowerBI. BAO is probably your best bet since they are at least covering the aggregate connection with powerBI and have loads of experience in tracker. Do please post here what you/they come up with. You might be the first, but I’m sure you won’t be the last to face this issue. : )
On the other issues, do you run into analytics performance issue on the DHIS2 side, or it is just in the push to powerBI? On the DE limit to 1600, there is not a practical limit to the number of data element allowed in DHIS2. Most large instances have well over 1600 DEs. In fact it’s not uncommon to find 30,000 - 50,000 DEs in some HMISs. Having that many DEs can present a bit of a meta-data organization nightmare, but otherwise its perfectly acceptable. When you try to go over 1600 what is the issue you are seeing? Are you being blocked somehow or is it failing to save?
@Markus might know more about limited to data elements in a single tracker program, but again I do not think 1600 should be an issue.
1 Like
Thanks for this Scott; knowing there are 30,000 DE systems is fantastic.
The 1600 DE error is from postgres analytics limit limit perhaps @Markus might advise on this aspect specifically >
when we ran DHIS2 analytics after we had imported a batch of 1500 new data elements. The error was 8 pages long, referenced hundreds of DE unique IDs, and ended with “ERROR: tables can have at most 1600 columns; nested exception is org.postgresql.util.PSQLException: ERROR: tables can have at most 1600 columns”
1 Like
Hmm . . . .Were you also importing the DE UIDs? If you were there is a specific format necessary for them that we see folks often having issues with. Usually the easiest course is to not include any UID in the import and just let DHIS2 create them automatically for you.
1 Like