Hi to all.
I want to test the ETL process with DHIS2. Is there an automated way to do this? is there a recommended tiers soft to use for this? my operational DB to use are excel files.
Thank you in advance.
Henri Moungui
Hi Henri,
There is no specific 3rd party software that you have to use, people use many different tools. Perhaps most commonly, people use scripts in Python or R or similar to load data. You can import both metadata and data in CSV format, for example.
http://dhis2.github.io/dhis2-docs/2.21/en/developer/html/ch01s10.html
http://dhis2.github.io/dhis2-docs/2.21/en/user/html/ch21.html
Knut
···
On Mon, Mar 21, 2016 at 3:15 PM, henri claude moungui henrimoungui@yahoo.fr wrote:
Hi to all.
I want to test the ETL process with DHIS2. Is there an automated way to do this? is there a recommended tiers soft to use for this? my operational DB to use are excel files.
Thank you in advance.
Henri Moungui
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp
–
Knut Staring
Dept. of Informatics, University of Oslo
Norway: +4791880522
Skype: knutstar
Thank you Knut.
Henri
Hi Henri,
There is no specific 3rd party software that you have to use, people use many different tools. Perhaps most commonly, people use scripts in Python or R or similar to load data. You can import both metadata and data in CSV format, for example.
http://dhis2.github.io/dhis2-docs/2.21/en/developer/html/ch01s10.html
http://dhis2.github.io/dhis2-docs/2.21/en/user/html/ch21.html
Knut
···
Le Lundi 21 mars 2016 15h25, Knut Staring knutst@gmail.com a écrit :
On Mon, Mar 21, 2016 at 3:15 PM, henri claude moungui henrimoungui@yahoo.fr wrote:
Hi to all.
I want to test the ETL process with DHIS2. Is there an automated way to do this? is there a recommended tiers soft to use for this? my operational DB to use are excel files.
Thank you in advance.
Henri Moungui
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp
–
Knut Staring
Dept. of Informatics, University of Oslo
Norway: +4791880522
Skype: knutstar
Hi Henri,
As Knut says, there is no “automatic” way of doing this, mostly because your Excel files are likely completely custom. Thus, you will need to transform your Excel files into something which DHIS2 can understand, possibly using CSV to import data.
I have developed a somewhat more technical example of how this can be accomplished with the R programming language, although regardless of you what use, the steps are basically the same.
You can find that here.
Regards,
Jason
···
On Mon, Mar 21, 2016 at 5:21 PM, henri claude moungui henrimoungui@yahoo.fr wrote:
Thank you Knut.
Henri
Le Lundi 21 mars 2016 15h25, Knut Staring knutst@gmail.com a écrit :
Hi Henri,
There is no specific 3rd party software that you have to use, people use many different tools. Perhaps most commonly, people use scripts in Python or R or similar to load data. You can import both metadata and data in CSV format, for example.
http://dhis2.github.io/dhis2-docs/2.21/en/developer/html/ch01s10.html
http://dhis2.github.io/dhis2-docs/2.21/en/user/html/ch21.html
Knut
On Mon, Mar 21, 2016 at 3:15 PM, henri claude moungui henrimoungui@yahoo.fr wrote:
Hi to all.
I want to test the ETL process with DHIS2. Is there an automated way to do this? is there a recommended tiers soft to use for this? my operational DB to use are excel files.
Thank you in advance.
Henri Moungui
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp
–
Knut Staring
Dept. of Informatics, University of Oslo
Norway: +4791880522
Skype: knutstar
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp
–
Jason P. Pickering
email: jason.p.pickering@gmail.com
tel:+46764147049
Hi Henri,
I read on this forum that someone was attempting to use Kettle (Pentaho Data Integration). I don’t know whether they succeeded.
Regards
Ant Snyman
This message and any attachments are subject to a disclaimer published at http://www.hisp.org/policies.html#comms_disclaimer. Please read the disclaimer before opening any attachment or taking any other action in terms of this electronic transmission. If you cannot access the disclaimer, kindly send an email to disclaimer@hisp.org and a copy will be provided to you. By replying to this e-mail or opening any attachment you agree to be bound by the provisions of the disclaimer.
···
On 21 March 2016 at 18:21, henri claude moungui henrimoungui@yahoo.fr wrote:
Thank you Knut.
Henri
Le Lundi 21 mars 2016 15h25, Knut Staring knutst@gmail.com a écrit :
Hi Henri,
There is no specific 3rd party software that you have to use, people use many different tools. Perhaps most commonly, people use scripts in Python or R or similar to load data. You can import both metadata and data in CSV format, for example.
http://dhis2.github.io/dhis2-docs/2.21/en/developer/html/ch01s10.html
http://dhis2.github.io/dhis2-docs/2.21/en/user/html/ch21.html
Knut
On Mon, Mar 21, 2016 at 3:15 PM, henri claude moungui henrimoungui@yahoo.fr wrote:
Hi to all.
I want to test the ETL process with DHIS2. Is there an automated way to do this? is there a recommended tiers soft to use for this? my operational DB to use are excel files.
Thank you in advance.
Henri Moungui
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp
Knut Staring
Dept. of Informatics, University of Oslo
Norway: +4791880522
Skype: knutstar
–
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp
–
Ant Snyman
Cell: 0824910449
Landline: 028 2713242
Health Information Systems Program - SA
Good point, Ant.
Here is a link to that conversation, with many interesting points:
https://lists.launchpad.net/dhis2-users/msg09234.html
Knut
···
On Tue, Mar 22, 2016 at 6:23 AM, Ant Snyman ant@hisp.org wrote:
Hi Henri,
I read on this forum that someone was attempting to use Kettle (Pentaho Data Integration). I don’t know whether they succeeded.
Regards
Ant Snyman
This message and any attachments are subject to a disclaimer published at http://www.hisp.org/policies.html#comms_disclaimer. Please read the disclaimer before opening any attachment or taking any other action in terms of this electronic transmission. If you cannot access the disclaimer, kindly send an email to disclaimer@hisp.org and a copy will be provided to you. By replying to this e-mail or opening any attachment you agree to be bound by the provisions of the disclaimer.
–
Knut Staring
Dept. of Informatics, University of Oslo
Norway: +4791880522
Skype: knutstar
On 21 March 2016 at 18:21, henri claude moungui henrimoungui@yahoo.fr wrote:
Thank you Knut.
Henri
Le Lundi 21 mars 2016 15h25, Knut Staring knutst@gmail.com a écrit :
Hi Henri,
There is no specific 3rd party software that you have to use, people use many different tools. Perhaps most commonly, people use scripts in Python or R or similar to load data. You can import both metadata and data in CSV format, for example.
http://dhis2.github.io/dhis2-docs/2.21/en/developer/html/ch01s10.html
http://dhis2.github.io/dhis2-docs/2.21/en/user/html/ch21.html
Knut
On Mon, Mar 21, 2016 at 3:15 PM, henri claude moungui henrimoungui@yahoo.fr wrote:
Hi to all.
I want to test the ETL process with DHIS2. Is there an automated way to do this? is there a recommended tiers soft to use for this? my operational DB to use are excel files.
Thank you in advance.
Henri Moungui
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp
Knut Staring
Dept. of Informatics, University of Oslo
Norway: +4791880522
Skype: knutstar
–
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp
–
Ant Snyman
Cell: 0824910449
Landline: 028 2713242
Health Information Systems Program - SA
Hi - we did some work using Talend based on client desires. We found it to be an incredible pain in the butt. Although our group are typically senior software developers or technical architects we had to pull in vendor consultants to get training on how to use it and ultimately dumped it. Beware the marketing - they all promote self-service but I suspect you’ll either be bringing in their trainers, hiring their consultants or finding your own software developers to actually get the job done.
Aaron
···
On Tue, Mar 22, 2016 at 12:04 AM, Knut Staring knutst@gmail.com wrote:
Good point, Ant.
Here is a link to that conversation, with many interesting points:
Knut
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp
–
On Tue, Mar 22, 2016 at 6:23 AM, Ant Snyman ant@hisp.org wrote:
Hi Henri,
I read on this forum that someone was attempting to use Kettle (Pentaho Data Integration). I don’t know whether they succeeded.
Regards
Ant Snyman
This message and any attachments are subject to a disclaimer published at http://www.hisp.org/policies.html#comms_disclaimer. Please read the disclaimer before opening any attachment or taking any other action in terms of this electronic transmission. If you cannot access the disclaimer, kindly send an email to disclaimer@hisp.org and a copy will be provided to you. By replying to this e-mail or opening any attachment you agree to be bound by the provisions of the disclaimer.
–
Knut Staring
Dept. of Informatics, University of Oslo
Norway: +4791880522
Skype: knutstar
On 21 March 2016 at 18:21, henri claude moungui henrimoungui@yahoo.fr wrote:
Thank you Knut.
Henri
Le Lundi 21 mars 2016 15h25, Knut Staring knutst@gmail.com a écrit :
Hi Henri,
There is no specific 3rd party software that you have to use, people use many different tools. Perhaps most commonly, people use scripts in Python or R or similar to load data. You can import both metadata and data in CSV format, for example.
http://dhis2.github.io/dhis2-docs/2.21/en/developer/html/ch01s10.html
http://dhis2.github.io/dhis2-docs/2.21/en/user/html/ch21.html
Knut
On Mon, Mar 21, 2016 at 3:15 PM, henri claude moungui henrimoungui@yahoo.fr wrote:
Hi to all.
I want to test the ETL process with DHIS2. Is there an automated way to do this? is there a recommended tiers soft to use for this? my operational DB to use are excel files.
Thank you in advance.
Henri Moungui
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp
Knut Staring
Dept. of Informatics, University of Oslo
Norway: +4791880522
Skype: knutstar
–
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp
–
Ant Snyman
Cell: 0824910449
Landline: 028 2713242
Health Information Systems Program - SA
Aaron Gladders, 2Paths
http://www.2paths.com/
Hi.
Thank you Ant, Knut, Jason and Aaron for the tips. I’m sure the discussions we are having will make us continue to improve the DHIS2 platform, particularly for datawarehouse implementations.
I was also looking for Kettle and Talend for this purpose. Finally, I found DHIS2 very comfortable for Export and Import as it is possible to do this step by step (you choose which metadata you want to export/import).
Now I see the way is how to TRANSFORM the Excel databases to fit with the exported file. For now, I intend to do this manually…
Thank you.
Hi - we did some work using Talend based on client desires. We found it to be an incredible pain in the butt. Although our group are typically senior software developers or technical architects we had to pull in vendor consultants to get training on how to use it and ultimately dumped it. Beware the marketing - they all promote self-service but I suspect you’ll either be bringing in their trainers, hiring their consultants or finding your own software developers to actually get the job done.
Aaron
···
Le Mardi 22 mars 2016 7h58, Aaron Gladders agladders@2paths.com a écrit :
On Tue, Mar 22, 2016 at 12:04 AM, Knut Staring knutst@gmail.com wrote:
Good point, Ant.
Here is a link to that conversation, with many interesting points:
Knut
On Tue, Mar 22, 2016 at 6:23 AM, Ant Snyman ant@hisp.org wrote:
Hi Henri,
I read on this forum that someone was attempting to use Kettle (Pentaho Data Integration). I don’t know whether they succeeded.
Regards
Ant Snyman
On 21 March 2016 at 18:21, henri claude moungui henrimoungui@yahoo.fr wrote:
Thank you Knut.
Henri
Le Lundi 21 mars 2016 15h25, Knut Staring knutst@gmail.com a écrit :
Hi Henri,
There is no specific 3rd party software that you have to use, people use many different tools. Perhaps most commonly, people use scripts in Python or R or similar to load data. You can import both metadata and data in CSV format, for example.
http://dhis2.github.io/dhis2-docs/2.21/en/developer/html/ch01s10.html
http://dhis2.github.io/dhis2-docs/2.21/en/user/html/ch21.html
Knut
On Mon, Mar 21, 2016 at 3:15 PM, henri claude moungui henrimoungui@yahoo.fr wrote:
Hi to all.
I want to test the ETL process with DHIS2. Is there an automated way to do this? is there a recommended tiers soft to use for this? my operational DB to use are excel files.
Thank you in advance.
Henri Moungui
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp
–
Knut Staring
Dept. of Informatics, University of Oslo
Norway: +4791880522
Skype: knutstar
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp
–
Ant Snyman
Cell: 0824910449
Landline: 028 2713242
Health Information Systems Program - SA
This message and any attachments are subject to a disclaimer published at http://www.hisp.org/policies.html#comms_disclaimer. Please read the disclaimer before opening any attachment or taking any other action in terms of this electronic transmission. If you cannot access the disclaimer, kindly send an email to disclaimer@hisp.org and a copy will be provided to you. By replying to this e-mail or opening any attachment you agree to be bound by the provisions of the disclaimer.
–
Knut Staring
Dept. of Informatics, University of Oslo
Norway: +4791880522
Skype: knutstar
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp
–
Aaron Gladders, 2Paths
http://www.2paths.com/
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp
That is often the easiest way. Excel also comes with recordable macros and its own scripting language, VBA.
Good luck!
···
On Wed, Mar 23, 2016 at 9:01 AM, henri claude moungui henrimoungui@yahoo.fr wrote:
Hi.
Thank you Ant, Knut, Jason and Aaron for the tips. I’m sure the discussions we are having will make us continue to improve the DHIS2 platform, particularly for datawarehouse implementations.
I was also looking for Kettle and Talend for this purpose. Finally, I found DHIS2 very comfortable for Export and Import as it is possible to do this step by step (you choose which metadata you want to export/import).
Now I see the way is how to TRANSFORM the Excel databases to fit with the exported file. For now, I intend to do this manually…
Thank you.
Le Mardi 22 mars 2016 7h58, Aaron Gladders agladders@2paths.com a écrit :
Hi - we did some work using Talend based on client desires. We found it to be an incredible pain in the butt. Although our group are typically senior software developers or technical architects we had to pull in vendor consultants to get training on how to use it and ultimately dumped it. Beware the marketing - they all promote self-service but I suspect you’ll either be bringing in their trainers, hiring their consultants or finding your own software developers to actually get the job done.
Aaron
On Tue, Mar 22, 2016 at 12:04 AM, Knut Staring knutst@gmail.com wrote:
Good point, Ant.
Here is a link to that conversation, with many interesting points:
Knut
On Tue, Mar 22, 2016 at 6:23 AM, Ant Snyman ant@hisp.org wrote:
Hi Henri,
I read on this forum that someone was attempting to use Kettle (Pentaho Data Integration). I don’t know whether they succeeded.
Regards
Ant Snyman
On 21 March 2016 at 18:21, henri claude moungui henrimoungui@yahoo.fr wrote:
Thank you Knut.
Henri
Le Lundi 21 mars 2016 15h25, Knut Staring knutst@gmail.com a écrit :
Hi Henri,
There is no specific 3rd party software that you have to use, people use many different tools. Perhaps most commonly, people use scripts in Python or R or similar to load data. You can import both metadata and data in CSV format, for example.
http://dhis2.github.io/dhis2-docs/2.21/en/developer/html/ch01s10.html
http://dhis2.github.io/dhis2-docs/2.21/en/user/html/ch21.html
Knut
On Mon, Mar 21, 2016 at 3:15 PM, henri claude moungui henrimoungui@yahoo.fr wrote:
Hi to all.
I want to test the ETL process with DHIS2. Is there an automated way to do this? is there a recommended tiers soft to use for this? my operational DB to use are excel files.
Thank you in advance.
Henri Moungui
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp
–
Knut Staring
Dept. of Informatics, University of Oslo
Norway: +4791880522
Skype: knutstar
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp
–
Ant Snyman
Cell: 0824910449
Landline: 028 2713242
Health Information Systems Program - SA
This message and any attachments are subject to a disclaimer published at http://www.hisp.org/policies.html#comms_disclaimer. Please read the disclaimer before opening any attachment or taking any other action in terms of this electronic transmission. If you cannot access the disclaimer, kindly send an email to disclaimer@hisp.org and a copy will be provided to you. By replying to this e-mail or opening any attachment you agree to be bound by the provisions of the disclaimer.
–
Knut Staring
Dept. of Informatics, University of Oslo
Norway: +4791880522
Skype: knutstar
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp
–
Aaron Gladders, 2Paths
http://www.2paths.com/
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp
–
Knut Staring
Dept. of Informatics, University of Oslo
Norway: +4791880522
Skype: knutstar
HI Henri,
sorry for for chipping in late, I am on vaccation. We are using kettle for
extracting data into DHIS2 as a prototype for a datawarehouse in a corporate
setting. A few of Aarons points on Talend apply for kettle as well. Things can
get tricky, but that is partly because of the nature of data import into DHIS2.
Support, community etc are not as vivid as the DHIS2 community, but you still
find some material on the net.
It runs and does the job. Currently we are batteling with memory issues as I am
trying to find some suitable method of packaging data partitions. Again, that is
not only kettles fault.
I just found out that the MoH is also using kettle in their HMIS setting, so I
am quite confident to have a few ressource persons in the country for
sustainable hand-over.
Compared to R etc I like the graphical data-flow, which helps when discussing
transformations with users. You can actually see what is going into and is
coming out of a process with a mouse-click.
Our current transformations are quite generique for extracting from a custom
data-source, hoping to put them on the net after some clean ups.
Generally speaking I'd prefere to have more extensive ETL capabilites within
DHIS2, though, e.g. to better synchronise and monitor load-jobs, not having to
allocate memory to two java engines, consistent management of data-elements
etc...
Regards,
Uwe
···
henri claude moungui <henrimoungui@yahoo.fr> hat am 23. März 2016 um 11:01
geschrieben:Hi.Thank you Ant, Knut, Jason and Aaron for the tips. I'm sure the discussions
we are having will make us continue to improve the DHIS2 platform,
particularly for datawarehouse implementations.I was also looking for Kettle
and Talend for this purpose. Finally, I found DHIS2 very comfortable for
Export and Import as it is possible to do this step by step (you choose which
metadata you want to export/import). Now I see the way is how to TRANSFORM the
Excel databases to fit with the exported file. For now, I intend to do this
manually...
Thank you.Le Mardi 22 mars 2016 7h58, Aaron Gladders <agladders@2paths.com> a écrit
:Hi - we did some work using Talend based on client desires. We found it to
be an incredible pain in the butt. Although our group are typically senior
software developers or technical architects we had to pull in vendor
consultants to get training on how to use it and ultimately dumped it. Beware
the marketing - they all promote self-service but I suspect you'll either be
bringing in their trainers, hiring their consultants or finding your own
software developers to actually get the job done.
AaronOn Tue, Mar 22, 2016 at 12:04 AM, Knut Staring <knutst@gmail.com> wrote:
Good point, Ant.
Here is a link to that conversation, with many interesting
points:Re: Importing DHS survey data in DHIS : Mailing list archive : dhis2-users team in LaunchpadKnut
On Tue, Mar 22, 2016 at 6:23 AM, Ant Snyman <ant@hisp.org> wrote:Hi Henri,I read on this forum that someone was attempting to use Kettle
(Pentaho Data Integration). I don't know whether they succeeded.RegardsAnt
Snyman
On 21 March 2016 at 18:21, henri claude moungui <henrimoungui@yahoo.fr> wrote:Thank you Knut.
HenriLe Lundi 21 mars 2016 15h25, Knut Staring <knutst@gmail.com> a écrit :
Hi Henri,
There is no specific 3rd party software that you have to use, people use many
different tools. Perhaps most commonly, people use scripts in Python or R or
similar to load data. You can import both metadata and data in CSV format, for
example.
http://dhis2.github.io/dhis2-docs/2.21/en/developer/html/ch01s10.html
http://dhis2.github.io/dhis2-docs/2.21/en/user/html/ch21.htmlKnut
On Mon, Mar 21, 2016 at 3:15 PM, henri claude moungui <henrimoungui@yahoo.fr> > wrote:Hi to all.I want to test the ETL process with DHIS2. Is there an automated way
to do this? is there a recommended tiers soft to use for this? my operational
DB to use are excel files.Thank you in advance.
Henri Moungui_______________________________________________
Mailing list: DHIS 2 Users in Launchpad
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : DHIS 2 Users in Launchpad
More help : https://help.launchpad.net/ListHelp--
Knut StaringDept. of Informatics, University of OsloNorway: +4791880522
Skype: knutstarhttp://dhis2.org_______________________________________________
Mailing list: DHIS 2 Users in Launchpad
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : DHIS 2 Users in Launchpad
More help : https://help.launchpad.net/ListHelp--
Ant SnymanCell: 0824910449Landline: 028 2713242
Health Information Systems Program - SA
This message and any attachments are subject to a disclaimer publishedat
http://www.hisp.org/policies.html#comms_disclaimer. Please read the disclaimer
before opening any attachment or taking any otheraction in terms of this
electronic transmission. If you cannot access thedisclaimer, kindly send an
email to disclaimer@hisp.org and a copy will be provided to you. Byreplying to
this e-mail or opening any attachment you agree to be bound by theprovisions
of the disclaimer.--
Knut StaringDept. of Informatics, University of OsloNorway: +4791880522
Skype: knutstarhttp://dhis2.org
_______________________________________________
Mailing list: DHIS 2 Users in Launchpad
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : DHIS 2 Users in Launchpad
More help : https://help.launchpad.net/ListHelp--
Aaron Gladders, 2Paths
http://www.2paths.com/
_______________________________________________
Mailing list: DHIS 2 Users in Launchpad
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : DHIS 2 Users in Launchpad
More help : https://help.launchpad.net/ListHelp_______________________________________________
Mailing list: DHIS 2 Users in Launchpad
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : DHIS 2 Users in Launchpad
More help : https://help.launchpad.net/ListHelp
Thank you Uwe for sharing your experience. Better late than never 
I agree with you. It is effectively the graphical data-flow on ETL that is missing on DHIS2…
Regards.
Henri
HI Henri,
sorry for for chipping in late, I am on vaccation. We are using kettle for
extracting data into DHIS2 as a prototype for a datawarehouse in a corporate
setting. A few of Aarons points on Talend apply for kettle as well. Things can
get tricky, but that is partly because of the nature of data import into DHIS2.
Support, community etc are not as vivid as the DHIS2 community, but you still
find some material on the net.
It runs and does the job. Currently we are batteling with memory issues as I am
trying to find some suitable method of packaging data partitions. Again, that is
not only kettles fault.
I just found out that the MoH is also using kettle in their HMIS setting, so I
am quite confident to have a few ressource persons in the country for
sustainable hand-over.
Compared to R etc I like the graphical data-flow, which helps when discussing
transformations with users. You can actually see what is going into and is
coming out of a process with a mouse-click.
Our current transformations are quite generique for extracting from a custom
data-source, hoping to put them on the net after some clean ups.
Generally speaking I’d prefere to have more extensive ETL capabilites within
DHIS2, though, e.g. to better synchronise and monitor load-jobs, not having to
allocate memory to two java engines, consistent management of data-elements
etc…
Regards,
Uwe
···
Le Jeudi 24 mars 2016 9h18, Uwe Wahser uwe@wahser.de a écrit :
henri claude moungui henrimoungui@yahoo.fr hat am 23. März 2016 um 11:01
geschrieben:
Hi.Thank you Ant, Knut, Jason and Aaron for the tips. I’m sure the discussions
we are having will make us continue to improve the DHIS2 platform,
particularly for datawarehouse implementations.I was also looking for Kettle
and Talend for this purpose. Finally, I found DHIS2 very comfortable for
Export and Import as it is possible to do this step by step (you choose which
metadata you want to export/import). Now I see the way is how to TRANSFORM the
Excel databases to fit with the exported file. For now, I intend to do this
manually…
Thank you.Le Mardi 22 mars 2016 7h58, Aaron Gladders agladders@2paths.com a écrit
:Hi - we did some work using Talend based on client desires. We found it to
be an incredible pain in the butt. Although our group are typically senior
software developers or technical architects we had to pull in vendor
consultants to get training on how to use it and ultimately dumped it. Beware
the marketing - they all promote self-service but I suspect you’ll either be
bringing in their trainers, hiring their consultants or finding your own
software developers to actually get the job done.
AaronOn Tue, Mar 22, 2016 at 12:04 AM, Knut Staring knutst@gmail.com wrote:
Good point, Ant.
Here is a link to that conversation, with many interesting
points:https://lists.launchpad.net/dhis2-users/msg09234.htmlKnut
On Tue, Mar 22, 2016 at 6:23 AM, Ant Snyman ant@hisp.org wrote:Hi Henri,I read on this forum that someone was attempting to use Kettle
(Pentaho Data Integration). I don’t know whether they succeeded.RegardsAnt
Snyman
On 21 March 2016 at 18:21, henri claude moungui henrimoungui@yahoo.fr wrote:Thank you Knut.
HenriLe Lundi 21 mars 2016 15h25, Knut Staring knutst@gmail.com a écrit :
Hi Henri,
There is no specific 3rd party software that you have to use, people use many
different tools. Perhaps most commonly, people use scripts in Python or R or
similar to load data. You can import both metadata and data in CSV format, for
example.
http://dhis2.github.io/dhis2-docs/2.21/en/developer/html/ch01s10.html
http://dhis2.github.io/dhis2-docs/2.21/en/user/html/ch21.htmlKnut
On Mon, Mar 21, 2016 at 3:15 PM, henri claude moungui henrimoungui@yahoo.fr > wrote:Hi to all.I want to test the ETL process with DHIS2. Is there an automated way
to do this? is there a recommended tiers soft to use for this? my operational
DB to use are excel files.Thank you in advance.
Henri Moungui
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp–
Knut StaringDept. of Informatics, University of OsloNorway: +4791880522
Skype: knutstarhttp://dhis2.org
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp–
Ant SnymanCell: 0824910449Landline: 028 2713242
Health Information Systems Program - SA
This message and any attachments are subject to a disclaimer publishedat
[http://www.hisp.org/policies.html#comms_disclaimer. Please ](http://www.hisp.org/policies.html#comms_disclaimer. Please)read the disclaimer
before opening any attachment or taking any otheraction in terms of this
electronic transmission. If you cannot access thedisclaimer, kindly send an
email to disclaimer@hisp.org and a copy will be provided to you. Byreplying to
this e-mail or opening any attachment you agree to be bound by theprovisions
of the disclaimer.–
Knut StaringDept. of Informatics, University of OsloNorway: +4791880522
Skype: knutstarhttp://dhis2.org
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp–
Aaron Gladders, 2Paths
http://www.2paths.com/
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp
Mailing list: https://launchpad.net/~dhis2-users
Post to : dhis2-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-users
More help : https://help.launchpad.net/ListHelp