Data mart exports

Hi,
I have some questions about setting up data mart exports:

  1. Is it a bug that you have to select something in all fields when creating data marts (data elements AND indicators etc)?? For example, it’s not possible to use only relative periods at the moment, unless you also choose at least one fixed period. (This is on latest stable 2.5)

  2. I noticed this in the help section: “Due to the limitation in number of columns per table in the database there is a limit to how many data elements that can be selected per data mart export. In postgres this limot is 255 (need to verify this number) data elements.”

  • Is that the case? (Checked the dhis.uio.no/demo server, which had several with more than 250).
  • If so, can you have 250 data element AND 250 indicators, or is it the sum that counts?

Regards
Olav

  1. I noticed this in the help section: “Due to the limitation in number of columns per table in the database there is a limit to how many data elements that can be selected per data mart export. In postgres this limot is 255 (need to verify this number) data elements.”
  • Is that the case? (Checked the dhis.uio.no/demo server, which had several with more than 250).
  • If so, can you have 250 data element AND 250 indicators, or is it the sum that counts?

It is the sum which counts, and it depends on the column types and can be increased by increasing the block size of postgres and doing a manual recompile
This really depends on the actual number of cross-tab columns, which may be a product of data elements and periods for instance, or other cross-tab dimesions.

You may be able to get away with a larger number than 255, but likely not more than 1600 (the upper limit provided by the postgres devs) for a standard install. This could be increased through either a custom recompile, or more easily (and certainly more robust from a performance perspective) through table partitioning. However, this would require a lot of custom stuff to achieve I think, and would probably not be worth the effort. I would try and experiment and establish an upper limit. If this is still not acceptable, then you might want to try an custom build of postgres. I would assume there is also some sort of limit with hibernate but an not totally sure what this might be.

Regards,
Jason

···

Hi Olav, this limitation is now removed, crosstabulation happens
against an in-memory hsql database. Are you able to remove that from
the docs?

Re selection of data elements / indicators, you are correct its a bug, will fix.

Lars

OK, thanks. Will look at fixing the documentation.

Olav

···

Den 1. nov. 2011 kl. 15:00 skrev Lars Helge Øverland:

Hi Olav, this limitation is now removed, crosstabulation happens
against an in-memory hsql database. Are you able to remove that from
the docs?

Re selection of data elements / indicators, you are correct its a bug, will fix.

Lars