DXF <-> entity mapping

Support - Assistance technique
1

Hi,

moving to the list, as I have some general questions I want to discuss on the mapping (I have attached an example of the impact of annotating the entities directly with jaxb).

Ok not realy a mistake so much as a flight of fancy. I had been playing with the direction of these associations in order to optimize the ordering of elements.

Ok, I thought maybe this optimization might be necessary.

Just checked in revised version which follows the model more literally which will make mapping easier. And updated sample. And added some missing elements of orgunit.

Great.

As an aside, there are generally a lot of empty elements which can be quite costly if we are expressing a great many orgunits. I can probably make most of these optional ie. if there is no url (Null in database) there is no need to create <url/> tag. This would be particularly useful with the date elements. Currently they are typed text rather than date otherwise empty elements don't validate. I think its better to rather enforce the date type and allow the element to be optional. Any thoughts? There are lots of elements in the metadata (eg dataElement) where this is the case.

Personally I like less xml and more enforcing :slight_smile: It does mean a more job, though.

On the general questions:

I like the fact that the consequences of changes to entities for the xml mapping is visible on the entity itself. The domain model is generally complex enough that this direct visibility will help a lot in visualizing the concequences of changes. We can add things to the model without to much problems, and to a certain degree "generalize" the contents of existing elements. But we are enforcing a stabilization of the api, the underlying model can´t really change without rethinking the xml mapping. And the proposed approach is rather inflexible and doesn´t handle mulitple versions very well (if we need to support multiple xml formats). How much do we expect the domain model to change and needing to be "backwards compatible"?

The real question is a general one; how do we think the api (and the xml formats we want to support) will evolve in the future? I don´t think we can avoid the rising cost of changes to the domain model, but we should try to make a concious choice on where we think we are going, before I create a new mapping solution. Some possiblities:

1) Proposed jaxb mapping, generally not supporting multiple xml versions - visible, easy, but no support for old formats
2) Something like the existing "mapping layer" - low visibility, easier to support multiple separate xml versions (but rising uncertainty of entity changes for every version)
3) jaxb and "versioned" domain models - visible, reflecting the (real) complexity of different api versions in the domain model

I don´t think we are at a point yet where we are sold on the approach we choose now, but obviously I would prefer not have to rewrite the mapping approach in the near future. I haven´t really looked at the feasibility of trying to do something like 3, but I think it should be to introduce such a model on parts of the domain model when needed, starting from 1.

Personally I prefer to try 1, what do you think?

Jo

OrganisationUnitGroupSet.java (6.34 KB)

···

Den 15. des. 2009 kl. 01.59 skrev Bob Jolliffe:

2

Hi,

moving to the list, as I have some general questions I want to discuss on the mapping (I have attached an example of the impact of annotating the entities directly with jaxb).

Ok not realy a mistake so much as a flight of fancy. I had been playing with the direction of these associations in order to optimize the ordering of elements.

Ok, I thought maybe this optimization might be necessary.

Just checked in revised version which follows the model more literally which will make mapping easier. And updated sample. And added some missing elements of orgunit.

Great.

As an aside, there are generally a lot of empty elements which can be quite costly if we are expressing a great many orgunits. I can probably make most of these optional ie. if there is no url (Null in database) there is no need to create tag. This would be particularly useful with the date elements. Currently they are typed text rather than date otherwise empty elements don’t validate. I think its better to rather enforce the date type and allow the element to be optional. Any thoughts? There are lots of elements in the metadata (eg dataElement) where this is the case.

Personally I like less xml and more enforcing :slight_smile: It does mean a more job, though.

I will do that - but probably tomorrow.

On the general questions:

I like the fact that the consequences of changes to entities for the xml mapping is visible on the entity itself. The domain model is generally complex enough that this direct visibility will help a lot in visualizing the concequences of changes. We can add things to the model without to much problems, and to a certain degree “generalize” the contents of existing elements. But we are enforcing a stabilization of the api, the underlying model can´t really change without rethinking the xml mapping. And the proposed approach is rather inflexible and doesn´t handle mulitple versions very well (if we need to support multiple xml formats). How much do we expect the domain model to change and needing to be “backwards compatible”?

The real question is a general one; how do we think the api (and the xml formats we want to support) will evolve in the future? I don´t think we can avoid the rising cost of changes to the domain model, but we should try to make a concious choice on where we think we are going, before I create a new mapping solution. Some possiblities:

  1. Proposed jaxb mapping, generally not supporting multiple xml versions - visible, easy, but no support for old formats

I have for some time been proposing to Lars that the marshalling and unmarshalling of domain objects should be part of the API. There are lots of advantages but I have come around to his thinking that decoupling these is in fact important. Primarily because I think there will be ongoing tweaks to the domain model. Particularly I expect we will do at least one more refactor of mcdonalds (categories, combo’s etc).

  1. Something like the existing “mapping layer” - low visibility, easier to support multiple separate xml versions (but rising uncertainty of entity changes for every version)

  2. jaxb and “versioned” domain models - visible, reflecting the (real) complexity of different api versions in the domain model

I don´t think we are at a point yet where we are sold on the approach we choose now, but obviously I would prefer not have to rewrite the mapping approach in the near future. I haven´t really looked at the feasibility of trying to do something like 3, but I think it should be to introduce such a model on parts of the domain model when needed, starting from 1.

Personally I prefer to try 1, what do you think?

My thinking was some combination of 2 and 3. A jaxb set of marshalling/unmarshalling classes which are bound to a namespaced version. And then mapping these to the domain model. Not as neat as a direct jaxb mapping but more flexible.

Bob

···

2009/12/15 Jo Størset storset@gmail.com

Den 15. des. 2009 kl. 01.59 skrev Bob Jolliffe:

Jo

3

I´m not entirely convinced that it will be that much more flexible. I mean, it really depends on how important backwards compability will be, and currently the domain model is a bit too changing to warrant it. So, at the moment I tend to think that planned breaking changes to a single export format would be preferable. The added complexity and extra resources needed to maintain a separate mapping layer and multiple versions just seem to outweigh the benefits. If we don´t have that many concrete use cases in the short term, the cost of upgrading them seems preferable?

On another note, I don´t think jaxb makes a lot of sense if we go for a separate layer. I would tend to favour a solution more like the current framework, simpler and with less boilerplate overhead.

Jo

···

Den 15. des. 2009 kl. 10.54 skrev Bob Jolliffe:

The real question is a general one; how do we think the api (and the xml formats we want to support) will evolve in the future? I don´t think we can avoid the rising cost of changes to the domain model, but we should try to make a concious choice on where we think we are going, before I create a new mapping solution. Some possiblities:

  1. Proposed jaxb mapping, generally not supporting multiple xml versions - visible, easy, but no support for old formats

I have for some time been proposing to Lars that the marshalling and unmarshalling of domain objects should be part of the API. There are lots of advantages but I have come around to his thinking that decoupling these is in fact important. Primarily because I think there will be ongoing tweaks to the domain model. Particularly I expect we will do at least one more refactor of mcdonalds (categories, combo’s etc).

  1. Something like the existing “mapping layer” - low visibility, easier to support multiple separate xml versions (but rising uncertainty of entity changes for every version)

  2. jaxb and “versioned” domain models - visible, reflecting the (real) complexity of different api versions in the domain model

I don´t think we are at a point yet where we are sold on the approach we choose now, but obviously I would prefer not have to rewrite the mapping approach in the near future. I haven´t really looked at the feasibility of trying to do something like 3, but I think it should be to introduce such a model on parts of the domain model when needed, starting from 1.

Personally I prefer to try 1, what do you think?

My thinking was some combination of 2 and 3. A jaxb set of marshalling/unmarshalling classes which are bound to a namespaced version. And then mapping these to the domain model. Not as neat as a direct jaxb mapping but more flexible.

4

The real question is a general one; how do we think the api (and the xml formats we want to support) will evolve in the future? I don´t think we can avoid the rising cost of changes to the domain model, but we should try to make a concious choice on where we think we are going, before I create a new mapping solution. Some possiblities:

  1. Proposed jaxb mapping, generally not supporting multiple xml versions - visible, easy, but no support for old formats

I have for some time been proposing to Lars that the marshalling and unmarshalling of domain objects should be part of the API. There are lots of advantages but I have come around to his thinking that decoupling these is in fact important. Primarily because I think there will be ongoing tweaks to the domain model. Particularly I expect we will do at least one more refactor of mcdonalds (categories, combo’s etc).

  1. Something like the existing “mapping layer” - low visibility, easier to support multiple separate xml versions (but rising uncertainty of entity changes for every version)

  2. jaxb and “versioned” domain models - visible, reflecting the (real) complexity of different api versions in the domain model

I don´t think we are at a point yet where we are sold on the approach we choose now, but obviously I would prefer not have to rewrite the mapping approach in the near future. I haven´t really looked at the feasibility of trying to do something like 3, but I think it should be to introduce such a model on parts of the domain model when needed, starting from 1.

Personally I prefer to try 1, what do you think?

My thinking was some combination of 2 and 3. A jaxb set of marshalling/unmarshalling classes which are bound to a namespaced version. And then mapping these to the domain model. Not as neat as a direct jaxb mapping but more flexible.

I´m not entirely convinced that it will be that much more flexible. I mean, it really depends on how important backwards compability will be, and currently the domain model is a bit too changing to warrant it. So, at the moment I tend to think that planned breaking changes to a single export format would be preferable. The added complexity and extra resources needed to maintain a separate mapping layer and multiple versions just seem to outweigh the benefits. If we don´t have that many concrete use cases in the short term, the cost of upgrading them seems preferable?

OK. I would say go with what you are convinced by. We can always pre-process old versions on the way in to massage them into whatever is current if the need arises.

Regards
Bob

···

2009/12/15 Jo Størset storset@gmail.com

Den 15. des. 2009 kl. 10.54 skrev Bob Jolliffe:

On another note, I don´t think jaxb makes a lot of sense if we go for a separate layer. I would tend to favour a solution more like the current framework, simpler and with less boilerplate overhead.

Jo

5

I´m not entirely convinced that it will be that much more flexible. I mean, it really depends on how important backwards compability will be, and currently the domain model is a bit too changing to warrant it. So, at the moment I tend to think that planned breaking changes to a single export format would be preferable. The added complexity and extra resources needed to maintain a separate mapping layer and multiple versions just seem to outweigh the benefits. If we don´t have that many concrete use cases in the short term, the cost of upgrading them seems preferable?

OK. I would say go with what you are convinced by.

Hey, don´t “go with what you are convinced by” me :slight_smile: Seriously, it would be helpful to have some common ground on where we will be moving in the short/medium future. I haven´t really gotten a good feeling for how things move in dhis yet, so my conviction might be way off.

This kind of feels like an important choice, so it might pay off to get it right. It feels likely to create a “path-dependency” (this phd stuff is getting to me :), the way we start now is not likely to be reversed. But I would still like to have some proposal ready this week.

We can always pre-process old versions on the way in to massage them into whatever is current if the need arises.

Yes, we should be able to, shouldn´t we.

Lars, I guess you are the one with the strongest opinions on this. Are you on the beach? Anyone else?

Jo

···

Den 15. des. 2009 kl. 11.47 skrev Bob Jolliffe:

6

Lars, I guess you are the one with the strongest opinions on this. Are you on the beach? Anyone else?

Jo

Hi,

I’m sorry but I don’t know that much about JAXB. I can only say that the fact that we already have defined a schema and the need for backwards-compatibility seem to necessitate and justify a decoupling between xml parsing and the object model. And another point is that we should try to wrap this up as soon as possible… Integration is needed and there are lots of other issues we need to take on…

Lars