Large financial data values requiring BIGINT data type

Hi

We are having problems capturing, storing, exporting and importing data comprising large numbers - with an annual public health budget of around R 150 billion, SA has a considerable number of financial data values well above 2 billion. While we CAN find ways of inserting such values since the value field in the DataValue table is char var, they don’t process well and don’t export/import well (e.g. the large values are exported into CSV as exponential number strings, which in turn is rejected during import into another instance).

Most database including PostgreSQL supports a BIGINT data type (8 bytes), which would work for us. DHIS2 don’t support BIGINT as a data type via the UI, but:

Anybody know of a way to circumvent that?

What the code implications of adding another data type - does it require code changes all over the place, or is the code reasonably contained?

Regards

Calle

···

Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRIC

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg


Hi Calle,

could you try to export as XML and see how those numbers look there?

It could be that the conversion to scientific notation might be an excel feature rather than DHIS 2.

regards,

Lars

···

On Mon, Nov 9, 2015 at 9:41 PM, Calle Hedberg calle.hedberg@gmail.com wrote:

Hi

We are having problems capturing, storing, exporting and importing data comprising large numbers - with an annual public health budget of around R 150 billion, SA has a considerable number of financial data values well above 2 billion. While we CAN find ways of inserting such values since the value field in the DataValue table is char var, they don’t process well and don’t export/import well (e.g. the large values are exported into CSV as exponential number strings, which in turn is rejected during import into another instance).

Most database including PostgreSQL supports a BIGINT data type (8 bytes), which would work for us. DHIS2 don’t support BIGINT as a data type via the UI, but:

Anybody know of a way to circumvent that?

What the code implications of adding another data type - does it require code changes all over the place, or is the code reasonably contained?

Regards

Calle


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRIC

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg



Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Lars Helge Øverland

Lead developer, DHIS 2

University of Oslo

Skype: larshelgeoverland

http://www.dhis2.org

Hi Calle,

Support for scientific notation was removed in revision 20247. So if you are past this, you probably need to convert your numbers into just plain numbers (no scientific notation). After that, they will be handled like “double precision” numbers in the analytics.

Regards,

Jason

···

On Mon, Nov 9, 2015 at 9:41 PM, Calle Hedberg calle.hedberg@gmail.com wrote:

Hi

We are having problems capturing, storing, exporting and importing data comprising large numbers - with an annual public health budget of around R 150 billion, SA has a considerable number of financial data values well above 2 billion. While we CAN find ways of inserting such values since the value field in the DataValue table is char var, they don’t process well and don’t export/import well (e.g. the large values are exported into CSV as exponential number strings, which in turn is rejected during import into another instance).

Most database including PostgreSQL supports a BIGINT data type (8 bytes), which would work for us. DHIS2 don’t support BIGINT as a data type via the UI, but:

Anybody know of a way to circumvent that?

What the code implications of adding another data type - does it require code changes all over the place, or is the code reasonably contained?

Regards

Calle


Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRIC

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg



Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+46764147049