dhis2 dxf data import

bobj · 15 September 2011 11:36

=0AThanks Bob, the pdf is useful.

When you mean codes, you mean the unique id for the record?
=0A
I wish life were so simple :-)=A0 There are quite a few ways that an identi=
fiable object (eg Orgunit) can be judged unique:

1.=A0 the primary database key=20
2.=A0 the name
3.=A0 the uuid
4.=A0 the code
=0A
1 and 2 are not good for a number of reasons.

3 is quite ok except that (a) its a bit long and (b) we might have to map t=
o data from elsewhere which doesn't use a uuid.

This latter case is quite common - if dhis was the central authority in the=
world for assigning metadata (sometimes it feels like it is designed as if=
it is life might be better - but the reality is that sometimes there a=
re other authorities and it is good that there are.=A0 The case we have bee=
n dealing with in Kenya for example - where they have an official Master Fa=
cility List which is responsible for registering facilities and assigning c=
odes.=A0 In which case we use these official codes in the code field of org=
unit.=A0=20
=0A=0A
Secondly, +1 for an internal routine to assign ids

I'm in two minds about this. =A0 For sure it might be better to have genera=
ted codes ou23, de456 etc rather than leave the field blank.=A0 But codes g=
enerally work best when assigned, such as the MFL case above.
=0A=0A
Thirdly, please how are you generating the dxf for ihris?=20

iHRIS is doing the generating the dxf for us ie. they are generating HR dat=
aelement values (number of docs, nurses etc)=20
=0A
Bob
=0A
Thanks

Ime

=0A=0A
=0A
This will also streamline metedata generation because one will only need to=
generate and pass metadata for only de/ou/period.

But I wonder; what's the difference between orgunitId and orgunit in your e=
xample.
=0A=0A

That's a typo.=A0 Please take a look at the pdf file I sent out earlier thi=
s week as that is more correct.
=A0
=0A=0AAlso, some elements don't use any categories, but the model reference=
s a default categorycombo. How will that look in your proposed schema?
=0A=0A
The default categorycombo is just that - default.=A0 So in the absence of a=
ny categories the categorycombo is automatically set to default when saving=
datavalues.=20
=0A=0A
Would you branch Jo's code in a way we could use easily test yours as a mod=
ule? or...
=0A=0A
The reading of this format is already implemented in the import/export modu=
le.=A0 It is tightly coupled with Jo's code in the sense of making use of t=
he same element/attribute name strings defined in his beans.=A0 So you can =
already use it by just importing the xml file.=A0 To test you should ideall=
y set up some codes in your database.=A0 We should try and do this in the d=
emo instance so people can try it there.=A0 Meanwhile I would suggest to te=
st:
=0A=0A
(i) pick an orgunit and assign it a code if it does not already have one (e=
g ou1)
(ii) pick a small dataset and assign it a code (eg dataset1)
(iii) assign codes to the dataelements within the dataset
(iv) assign the dataset to the orgunit
=0A=0A
Then you should be able to import datavaluesets according to the examples g=
iven.

Alternatively you can use the existing uuids instead of the codes.

(It might be worth having a startup routine which automatically assigns cod=
es based on the existing internal ids where they do not already exist.)
=0A=0A
Regards
Bob
=A0
=0A
Thanks

Ime

=0A=0AThe implication of adding all the above will be that whereas=0A the
datavalueset above will remain valid (except perhaps shifting to
storedBy), the following would also be valid:

I am pretty sure I can implement the above without breaking what is
=0A=0Acurrently there.=A0 One possible but minor breaking change I would
suggest to improving parsing of very large datasets might be to
abbreviate some well known element names to dv, de and v=0A for
compactness.

=0A
=0A
--270156429-2103316800-1316136964=:74243
Content-Type: text/html; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable

<table cellspacing=3D"0" cellpadding=3D"0" border=3D"0" ><tr><td valign=3D"=
top" style=3D"font: inherit;">Hi Bob,<br>Thanks for this.<br>This will be v=
ery useful.<br><br>what do you think about openmrs generating dxf like ihri=
s?<br><br>Ime<br><br>--- On <b>Thu, 9/15/11, Bob Jolliffe <i><bobjolliff=
e@gmail.com></i></b> wrote:<br><blockquote style=3D"border-left: 2px sol=
id rgb(16, 16, 255); margin-left: 5px; padding-left: 5px;"><br>From: Bob Jo=
lliffe <bobjolliffe@gmail.com><br>Subject: Re: [Dhis2-devs] dhis2 dxf=
data import<br>To: "Ime Asangansi" <asangansi@yahoo.com><br>Cc: "dhi=
s2-devs" <dhis2-devs@lists.launchpad.net><br>Date: Thursday, Septembe=
r 15, 2011, 11:36 AM<br><br><div id=3D"yiv344307900"><div class=3D"yiv34430=
7900gmail_quote">On 15 September 2011 10:01, Ime Asangansi <span dir=3D"ltr=
"><<a rel=3D"nofollow" ymailto=3D"mailto:asangansi@yahoo.com" target=3D"=
_blank" href=3D"/mc/compose?to=3Dasangansi@yahoo.com">asangansi@yahoo.com</=

></span>

wrote:<br><blockquote class=3D"yiv344307900gmail_quote" style=3D"margin: 0=
pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: =
1ex;">=0A<table border=3D"0" cellpadding=3D"0" cellspacing=3D"0"><tbody><tr=

<td style=3D"font: inherit;" valign=3D"top">Thanks Bob, the pdf is useful.=

<br><br>When you mean codes, you mean the unique id for the record?<br></td=

</tr></tbody></table>=0A</blockquote><div><br>I wish life were so simple :=

-)  There are quite a few ways that an identifiable object (eg Orgunit=
) can be judged unique:<br><br>1.  the primary database key <br>2.&nbs=
p; the name<br>3.  the uuid<br>4.  the code<br>=0A<br>1 and 2 are=
not good for a number of reasons.<br><br>3 is quite ok except that (a) its=
a bit long and (b) we might have to map to data from elsewhere which doesn=
't use a uuid.<br><br>This latter case is quite common - if dhis was the ce=
ntral authority in the world for assigning metadata (sometimes it feels lik=
e it is designed as if it is life might be better - but the reality is =
that sometimes there are other authorities and it is good that there are.&n=
bsp; The case we have been dealing with in Kenya for example - where they h=
ave an official Master Facility List which is responsible for registering f=
acilities and assigning codes.  In which case we use these official co=
des in the code field of orgunit.  <br>=0A</div><blockquote class=3D"y=
iv344307900gmail_quote" style=3D"margin: 0pt 0pt 0pt 0.8ex; border-left: 1p=
x solid rgb(204, 204, 204); padding-left: 1ex;"><table border=3D"0" cellpad=
ding=3D"0" cellspacing=3D"0"><tbody><tr><td style=3D"font: inherit;" valign=
=3D"top">=0A<br>Secondly, +1 for an internal routine to assign ids<br></td>=
</tr></tbody></table></blockquote><div><br>I'm in two minds about this. &nb=
sp; For sure it might be better to have generated codes ou23, de456 etc rat=
her than leave the field blank.  But codes generally work best when as=
signed, such as the MFL case above.<br>=0A</div><blockquote class=3D"yiv344=
307900gmail_quote" style=3D"margin: 0pt 0pt 0pt 0.8ex; border-left: 1px sol=
id rgb(204, 204, 204); padding-left: 1ex;"><table border=3D"0" cellpadding=
=3D"0" cellspacing=3D"0"><tbody><tr><td style=3D"font: inherit;" valign=3D"=
top">=0A<br>Thirdly, please how are you generating the dxf for ihris? <br><=
/td></tr></tbody></table></blockquote><div><br>iHRIS is doing the generatin=
g the dxf for us ie. they are generating HR dataelement values (number of d=
ocs, nurses etc) <br>=0A<br>Bob<br></div><blockquote class=3D"yiv344307900g=
mail_quote" style=3D"margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(=
204, 204, 204); padding-left: 1ex;"><table border=3D"0" cellpadding=3D"0" c=
ellspacing=3D"0"><tbody><tr><td style=3D"font: inherit;" valign=3D"top">=0A=
<br>Thanks<br><font color=3D"#888888"><br>Ime</font><div><div></div><div cl=
ass=3D"yiv344307900h5"><br><br><br>--- On <b>Thu, 9/15/11, Bob Jolliffe <i>=
<<a rel=3D"nofollow" ymailto=3D"mailto:bobjolliffe@gmail.com" target=3D"=
_blank" href=3D"/mc/compose?to=3Dbobjolliffe@gmail.com">bobjolliffe@gmail.c=
om</a>></i></b> wrote:<br>=0A<blockquote style=3D"border-left: 2px solid=
rgb(16, 16, 255); margin-left: 5px; padding-left: 5px;"><div><div><blockqu=
ote style=3D"margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204=
, 204); padding-left: 1ex;"><table border=3D"0" cellpadding=3D"0" cellspaci=
ng=3D"0">=0A<tbody><tr><td style=3D"font: inherit;" valign=3D"top"><br>=0A<=

This will also streamline metedata generation because one will only need=

to generate and pass metadata for only de/ou/period.<br><br>But I wonder; =
what's the difference between orgunitId and orgunit in your example.<br>=0A=
=0A</td></tr></tbody></table></blockquote><div><br><br>That's a typo. =
Please take a look at the pdf file I sent out earlier this week as that is=
more correct.<br> <br></div><blockquote style=3D"margin: 0pt 0pt 0pt =
0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">=0A=
=0A<table border=3D"0" cellpadding=3D"0" cellspacing=3D"0"><tbody><tr><td s=
tyle=3D"font: inherit;" valign=3D"top">Also, some elements don't use any ca=
tegories, but the model references a default categorycombo. How will that l=
ook in your proposed schema?<br>=0A=0A</td></tr></tbody></table></blockquot=

<div><br>The default categorycombo is just that - default.  So in th=

e absence of any categories the categorycombo is automatically set to defau=
lt when saving datavalues. <br></div><blockquote style=3D"margin: 0pt 0pt 0=
pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">=
=0A=0A<table border=3D"0" cellpadding=3D"0" cellspacing=3D"0"><tbody><tr><t=
d style=3D"font: inherit;" valign=3D"top"><br>Would you branch Jo's code in=
a way we could use easily test yours as a module? or...<br></td></tr></tbo=

</table>=0A=0A</blockquote><div><br>The reading of this format is alread=

y implemented in the import/export module.  It is tightly coupled with=
Jo's code in the sense of making use of the same element/attribute name st=
rings defined in his beans.  So you can already use it by just importi=
ng the xml file.  To test you should ideally set up some codes in your=
database.  We should try and do this in the demo instance so people c=
an try it there.  Meanwhile I would suggest to test:<br>=0A=0A<br>(i) =
pick an orgunit and assign it a code if it does not already have one (eg ou=
1)<br>(ii) pick a small dataset and assign it a code (eg dataset1)<br>(iii)=
assign codes to the dataelements within the dataset<br>(iv) assign the dat=
aset to the orgunit<br>=0A=0A<br>Then you should be able to import datavalu=
esets according to the examples given.<br><br>Alternatively you can use the=
existing uuids instead of the codes.<br><br>(It might be worth having a st=
artup routine which automatically assigns codes based on the existing inter=
nal ids where they do not already exist.)<br>=0A=0A<br>Regards<br>Bob<br>&n=
bsp;<br></div><blockquote style=3D"margin: 0pt 0pt 0pt 0.8ex; border-left: =
1px solid rgb(204, 204, 204); padding-left: 1ex;"><table border=3D"0" cellp=
adding=3D"0" cellspacing=3D"0"><tbody><tr>=0A<td style=3D"font: inherit;" v=
align=3D"top"><br>Thanks<br><font color=3D"#888888"><br>Ime</font><div><br>=
<br>--- On <b>Thu, 9/1/11, Bob Jolliffe <i><<a rel=3D"nofollow" target=
=3D"_blank" href=3D"http://mc/compose?to=3Dbobjolliffe@gmail.com">bobjollif=
fe@gmail.com</a>></i></b> wrote:<br>=0A=0A<blockquote style=3D"border-le=
ft: 2px solid rgb(16, 16, 255); margin-left: 5px; padding-left: 5px;">The i=
mplication of adding all the above will be that whereas=0A the<br><div>data=
valueset above will remain valid (except perhaps shifting to<br>storedBy), =
the following would also be valid:<br><br><dataValueSets orgUnitId=3D"co=
de" dataElementId=3D"internal"<br>  <dataValueSet  orgUnit=3D"=
23" period=3D"201004" storedBy=3D"Bob" ><br>=0A=0A    <data=
Value dataElement=3D"2" value=3D"4" Sex=3D"1" /><br>    <da=
taValue dataElement=3D"2" value=3D"5" Sex=3D"2"/><br>    <d=
ataValue dataElement=3D"4" value=3D"43" Sex=3D"1" Age=3D"3" /><br>=0A=0A=
    <dataValue dataElement=3D"5" value=3D"44" Sex=3D"1" Age=3D=
"3" /><br>  </dataValueSet><br></dataValueSets><br><br>=
I am pretty sure I can implement the above without breaking what is<br>=0A=
=0Acurrently there.  One possible but minor breaking change I would<br=

suggest to improving parsing of very large datasets might be to<br>abbrevi=

ate some well known element names to dv, de and v=0A for<br>compactness.</d=

<br></blockquote></div></td></tr></tbody></table></blockquote></div><br>=

=0A</div></blockquote></div></div></td></tr></tbody></table></blockquote></=

<br>=0A</div></blockquote></td></tr></table>

--270156429-2103316800-1316136964=:74243--

···

On 15 September 2011 10:01, Ime Asangansi <asangansi@yahoo.com> wrote:
--- On Thu, 9/15/11, Bob Jolliffe <bobjolliffe@gmail.com> wrote:
--- On Thu, 9/1/11, Bob Jolliffe <bobjolliffe@gmail.com> wrote: