Should we remove the Source object?

Hi,

I have been thinking about removing the Source object from the API.
The abstract Source class was introduced in the model in order to
allow for multiple implementations of Sources in the system. Currently
we only have the OrganisationUnit object extending Source. We now see
eg. with Abyot's CHIS module that even if we need a new kind of
Source; the Patient, we still need the OrganisationUnit reference in
DataValue.

The downsides of keeping Source are that it

- Increases complexity.
- Makes JDBC/Hibernate operations slower, since the Source table must
be updated first, then the orgunit table.
- Entails problems when people manually modifies the database (which
they really should not) as they always forget to update the source
table when adding/removing from the organisationunit table.

The only database update this would require is to set the
"organisationunitid" field in the "organisationunit" table to
auto-increment. I believe Hibernate will take care of this.

Based on this I opt for removing the Source object and moving the
OrganisationUnit object into the core.

Are there other aspects regarding this that should be mentioned? Any objections?

regards

Lars

Hi,

I support this proposed change.

As you say, it is most likely that we would have to keep a reference to an orgunit in any case, since most data relate to orgunits somehow, and then we would need another object, like the Patient. If it speeds up the application and simplifies things I think we should make the move in stead of waiting for something that has not come up over the last 4 years, and now seems even less likely to happen…

Ola

···

2009/8/26 Lars Helge Øverland larshelge@gmail.com

Hi,

I have been thinking about removing the Source object from the API.

The abstract Source class was introduced in the model in order to

allow for multiple implementations of Sources in the system. Currently

we only have the OrganisationUnit object extending Source. We now see

eg. with Abyot’s CHIS module that even if we need a new kind of

Source; the Patient, we still need the OrganisationUnit reference in

DataValue.

The downsides of keeping Source are that it

  • Increases complexity.

  • Makes JDBC/Hibernate operations slower, since the Source table must

be updated first, then the orgunit table.

  • Entails problems when people manually modifies the database (which

they really should not) as they always forget to update the source

table when adding/removing from the organisationunit table.

The only database update this would require is to set the

“organisationunitid” field in the “organisationunit” table to

auto-increment. I believe Hibernate will take care of this.

Based on this I opt for removing the Source object and moving the

OrganisationUnit object into the core.

Are there other aspects regarding this that should be mentioned? Any objections?

regards

Lars


Lars I think you are right. I think the household/person was the closest I think we got to considering deriving something else from source other than organisation unit.

I’d put it on notice - if you don’t get any suggested use cases before the weekend (or some other arbitrary point in time) then chop it. I don’t imagine it will have an enormous performance impact, but getting leaner is good.

Regards
Bob

···

2009/8/26 Lars Helge Øverland larshelge@gmail.com

Hi,

I have been thinking about removing the Source object from the API.

The abstract Source class was introduced in the model in order to

allow for multiple implementations of Sources in the system. Currently

we only have the OrganisationUnit object extending Source. We now see

eg. with Abyot’s CHIS module that even if we need a new kind of

Source; the Patient, we still need the OrganisationUnit reference in

DataValue.

The downsides of keeping Source are that it

  • Increases complexity.

  • Makes JDBC/Hibernate operations slower, since the Source table must

be updated first, then the orgunit table.

  • Entails problems when people manually modifies the database (which

they really should not) as they always forget to update the source

table when adding/removing from the organisationunit table.

The only database update this would require is to set the

“organisationunitid” field in the “organisationunit” table to

auto-increment. I believe Hibernate will take care of this.

Based on this I opt for removing the Source object and moving the

OrganisationUnit object into the core.

Are there other aspects regarding this that should be mentioned? Any objections?

regards

Lars


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Lars I think you are right. I think the household/person was the closest I
think we got to considering deriving something else from source other than
organisation unit.

I'd put it on notice - if you don't get any suggested use cases before the
weekend (or some other arbitrary point in time) then chop it. I don't
imagine it will have an enormous performance impact,

Thanks for the comments.

What made me reiterate this issue now was trying to import Jason's
Zambia database with 17 000 orgunits. We normally use multiple insert
sql statements to insert large amounts of data. Now for orgunits we
must first insert a record in the source table, retrieve the generated
id, then add an entry to the organisationunit table, which is way
slower than simply doing multiple inserts...

but getting leaner is good.

Agree

···

2009/8/26 Bob Jolliffe <bobjolliffe@gmail.com>:

Lars I think you are right. I think the household/person was the closest I

think we got to considering deriving something else from source other than

organisation unit.

I’d put it on notice - if you don’t get any suggested use cases before the

weekend (or some other arbitrary point in time) then chop it. I don’t

imagine it will have an enormous performance impact,

Thanks for the comments.

What made me reiterate this issue now was trying to import Jason’s

Zambia database with 17 000 orgunits. We normally use multiple insert

sql statements to insert large amounts of data. Now for orgunits we

must first insert a record in the source table, retrieve the generated

id, then add an entry to the organisationunit table, which is way

slower than simply doing multiple inserts…

Ola’s argument is good - agile principles say very clearly that you should not try and prepare for all kinds of future eventualities. Chuck it out without hesitation.

Knut

···

2009/8/26 Lars Helge Øverland larshelge@gmail.com

2009/8/26 Bob Jolliffe bobjolliffe@gmail.com:

but getting leaner is good.

Agree


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp


Cheers,
Knut Staring

Hi,

···

----- Original Message ----
From: Lars Helge Øverland <larshelge@gmail.com>
To: DHIS 2 developers <dhis2-devs@lists.launchpad.net>
Sent: Wednesday, August 26, 2009 6:56:46 PM
Subject: [Dhis2-devs] Should we remove the Source object?

Hi,

I have been thinking about removing the Source object from the API.
The abstract Source class was introduced in the model in order to
allow for multiple implementations of Sources in the system. Currently
we only have the OrganisationUnit object extending Source. We now see
eg. with Abyot's CHIS module that even if we need a new kind of
Source; the Patient, we still need the OrganisationUnit reference in
DataValue.

The downsides of keeping Source are that it

- Increases complexity.
- Makes JDBC/Hibernate operations slower, since the Source table must
be updated first, then the orgunit table.
- Entails problems when people manually modifies the database (which
they really should not) as they always forget to update the source
table when adding/removing from the organisationunit table.

The only database update this would require is to set the
"organisationunitid" field in the "organisationunit" table to
auto-increment. I believe Hibernate will take care of this.

Based on this I opt for removing the Source object and moving the
OrganisationUnit object into the core.

Are there other aspects regarding this that should be mentioned? Any objections?

regards

Lars

Just looked at Source class, it has some commonly used attributes for hibernated objects (needed by hibernated objects, not any business logic at all), it does not give any value to organisationunit class. Moreover its ID attribute and its storage to database is excessive. Useful object to me for example could be relation to resolve hierarchy, if say source class had attributes parent and childs, and other implementing classes could use it reducing code lines and maintenance, bringing all into interface:

private Object parent;
private List<Object> children;

gettersetters. This could be good for implementations with hierarchy on their business logic.

I agree with removal as it does nothing, but one precaution -
if it was used by modules as object via hibernate, those need fixing too. Look at datavalue class, it has Source class as attribute, maybe this is due to lightweightness of Source compared to Organisationunit? We have more load on datavalue object read and manipulation.

regards,
murod
_______________________________________________
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help : https://help.launchpad.net/ListHelp

Hi,

I have been thinking about removing the Source object from the API.

The abstract Source class was introduced in the model in order to

allow for multiple implementations of Sources in the system. Currently

we only have the OrganisationUnit object extending Source. We now see

eg. with Abyot’s CHIS module that even if we need a new kind of

Source; the Patient, we still need the OrganisationUnit reference in

DataValue.

I am not using it !

···

2009/8/26 Lars Helge Øverland larshelge@gmail.com

The downsides of keeping Source are that it

  • Increases complexity.

  • Makes JDBC/Hibernate operations slower, since the Source table must

be updated first, then the orgunit table.

  • Entails problems when people manually modifies the database (which

they really should not) as they always forget to update the source

table when adding/removing from the organisationunit table.

The only database update this would require is to set the

“organisationunitid” field in the “organisationunit” table to

auto-increment. I believe Hibernate will take care of this.

Based on this I opt for removing the Source object and moving the

OrganisationUnit object into the core.

Are there other aspects regarding this that should be mentioned? Any objections?

regards

Lars


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

OK but in all your design proposals there is a PatientDataValue object
with a reference to Source, meaning that you don't reuse the DataValue
object and associate it with a Patient Source implementation.

This is the last I've seen:

http://mail.google.com/mail/?ui=2&ik=b4dafba814&view=att&th=121c110ad771e794&attid=0.1&disp=inline&realattid=f_fvpi020h1&zw

···

2009/8/26 Abyot Gizaw <abyota@gmail.com>:

2009/8/26 Lars Helge Øverland <larshelge@gmail.com>

Hi,

I have been thinking about removing the Source object from the API.
The abstract Source class was introduced in the model in order to
allow for multiple implementations of Sources in the system. Currently
we only have the OrganisationUnit object extending Source. We now see
eg. with Abyot's CHIS module that even if we need a new kind of
Source; the Patient, we still need the OrganisationUnit reference in
DataValue.

I am not using it !

Yes that was the initial design … but now with employing OpenMRS for patient value storage that will be obsolete. If back to that design … then I can refer OrgUnit.

···

2009/8/26 Lars Helge Øverland larshelge@gmail.com

2009/8/26 Abyot Gizaw abyota@gmail.com:

2009/8/26 Lars Helge Øverland larshelge@gmail.com

Hi,

I have been thinking about removing the Source object from the API.

The abstract Source class was introduced in the model in order to

allow for multiple implementations of Sources in the system. Currently

we only have the OrganisationUnit object extending Source. We now see

eg. with Abyot’s CHIS module that even if we need a new kind of

Source; the Patient, we still need the OrganisationUnit reference in

DataValue.

I am not using it !

OK but in all your design proposals there is a PatientDataValue object

with a reference to Source, meaning that you don’t reuse the DataValue

object and associate it with a Patient Source implementation.

This is the last I’ve seen:

http://mail.google.com/mail/?ui=2&ik=b4dafba814&view=att&th=121c110ad771e794&attid=0.1&disp=inline&realattid=f_fvpi020h1&zw

Yes that was the initial design ...... but now with employing OpenMRS for
patient value storage that will be obsolete. If back to that design ... then
I can refer OrgUnit.

OK. Could you elaborate a little on how you are representing
DataValues? Anyway if you are using the Patient object in OpenMRS
extending Source and re-using DataValue will be out of the question.

Yes that is what I am saying… That design is obsolete making no datavalue in the CBHIS.

For storing patient values I am using all openmrs objects — Encounter --> Observaations --> Concept -->Value

Finally the question boils down to OpenMRS - DHIS integration. Which I think is another topic.

···

2009/8/26 Lars Helge Øverland larshelge@gmail.com

Yes that was the initial design … but now with employing OpenMRS for

patient value storage that will be obsolete. If back to that design … then

I can refer OrgUnit.

OK. Could you elaborate a little on how you are representing

DataValues? Anyway if you are using the Patient object in OpenMRS

extending Source and re-using DataValue will be out of the question.

Yes that is what I am saying.... That design is obsolete making no datavalue
in the CBHIS.

Maybe I misinterpreted you a little. Fine...

Hi,

I have been thinking about removing the Source object from the API.

The abstract Source class was introduced in the model in order to

allow for multiple implementations of Sources in the system. Currently

we only have the OrganisationUnit object extending Source. We now see

eg. with Abyot’s CHIS module that even if we need a new kind of

Source; the Patient, we still need the OrganisationUnit reference in

DataValue.

I am not using it !

OK but in all your design proposals there is a PatientDataValue object

with a reference to Source, meaning that you don’t reuse the DataValue

object and associate it with a Patient Source implementation.

This is the last I’ve seen:

http://mail.google.com/mail/?ui=2&ik=b4dafba814&view=att&th=121c110ad771e794&attid=0.1&disp=inline&realattid=f_fvpi020h1&zw

Yes that was the initial design … but now with employing OpenMRS for patient value storage that will be obsolete. If back to that design … then I can refer OrgUnit.

I hadn’t realized you had finally decided to go this route. Different thread, but will be interesting to hear how you find it. Basically same approach as Saptarshi’s leprosy module?

Back to the source, there is one possible constraint I am thinking. Currently the thing about an orgunit which makes it a bit special is that it is part of the orgunit hierarchy. What happens (or does it ever happen?) when you want to import data from some ad hoc source outside of the hierarchy - maybe a survey for analysis or something like that? You either have to allow for this different kind of source or probably its better to just disallow it and use a new instance of database instead.

By removing source we will be saying unequivicably that:
(i) all datavalues are associated with an orgunit; and
(ii) that orgunit is part of a single hierarchy

That’s neat enough for me. And I think reflects current concrete experience.

Regards
Bob

···

2009/8/26 Abyot Gizaw abyota@gmail.com

2009/8/26 Lars Helge Øverland larshelge@gmail.com

2009/8/26 Abyot Gizaw abyota@gmail.com:

2009/8/26 Lars Helge Øverland larshelge@gmail.com


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Back to the source, there is one possible constraint I am thinking.
Currently the thing about an orgunit which makes it a bit special is that it
is part of the orgunit hierarchy. What happens (or does it ever happen?)
when you want to import data from some ad hoc source outside of the
hierarchy - maybe a survey for analysis or something like that? You either
have to allow for this different kind of source or probably its better to
just disallow it and use a new instance of database instead.

Not sure. But I don't believe we will gather data from places outside
the hierarchy. 1.4 has survey support implemented, where the survey
and its data are associated with an organisation unit.