[Branch ~dhis2-devs-core/dhis2/trunk] Rev 2851: Added spike for storing dataValueSets through a simple http post (see <dhis-root-url>/api/rpc)

DHIS_2_Users · 15 February 2011 06:59

Jo_Storset · 15 February 2011 10:14

Hi,

just commited a spike for simple(?) external posting of data values. This is a tricky area to design, so I'm hoping this can generate some discussions around it.

The immediate use case for this is an external server doing doing the data collection (mobile based), and then needing an easy way to post dataValueSets, with what I think is more detailed automated validation and feedback than the imp/exp would allow for..

You should be able to try it out by going to [1] (or correspondingly to your local instance) and follow the instructions on screen.

I've tried to promote dataSet and the corresponding dataValueSet as top level concepts, as I think this makes more sense than having isolated data values..

Also, I've created representation for dataValueSet, that is sort of dxf-like, but doesn't really make for the batch-oriented import/export dxf/sdmx is intended for.
It is possible to tweak the format to be more batch-friendly, but I'm not convinced the use cases for this vs. batch are really similar enough that it makes sense at the cost of simplicity. One of the things this solution allows for, btw, is use of json instead of xml.

I think we do want an api more in this direction than the current import/export stuff will allow for, but I'm not in any way sure this is the right way to start off. In other words, maybe going with a layer above a reworked version of the xsl/xml based imp/exp framework would be better overall?

Anyway, have a quick look and feed back
Jo

[1] http://dhis.uio.no/dev/api/rpc

···

Den 15. feb. 2011 kl. 12.29 skrev noreply@launchpad.net:

------------------------------------------------------------
revno: 2851
committer: Jo Størset <storset@gmail.com>
branch nick: dhis2
timestamp: Tue 2011-02-15 12:23:26 +0530
message:
Added spike for storing dataValueSets through a simple http post (see <dhis-root-url>/api/rpc)
added:
dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/mapping/IllegalArgumentExceptionMapper.java
dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/rpc/
dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/rpc/RPCResource.java
modified:
dhis-2/dhis-web/dhis-web-api/src/main/resources/META-INF/dhis/beans.xml

--
lp:dhis2
trunk : Code : DHIS

Your team DHIS 2 developers is subscribed to branch lp:dhis2.
To unsubscribe from this branch go to OpenID transaction in progress
=== added file 'dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/mapping/IllegalArgumentExceptionMapper.java'
--- dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/mapping/IllegalArgumentExceptionMapper.java 1970-01-01 00:00:00 +0000
+++ dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/mapping/IllegalArgumentExceptionMapper.java 2011-02-15 06:53:26 +0000
@@ -0,0 +1,23 @@
+package org.hisp.dhis.web.api.mapping;
+
+import javax.ws.rs.core.MediaType;
+import javax.ws.rs.core.Response;
+import javax.ws.rs.core.Response.Status;
+import javax.ws.rs.ext.ExceptionMapper;
+import javax.ws.rs.ext.Provider;
+
+import com.sun.jersey.spi.resource.Singleton;
+
+@Provider
+@Singleton
+public class IllegalArgumentExceptionMapper
+ implements ExceptionMapper<IllegalArgumentException>
+{
+
+ @Override
+ public Response toResponse( IllegalArgumentException e )
+ {
+ return Response.status( Status.CONFLICT ).entity( "Problem with input: " + e.getMessage() ).type( MediaType.TEXT_PLAIN ).build();
+ }
+
+}

=== added directory 'dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/rpc'
=== added file 'dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/rpc/RPCResource.java'
--- dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/rpc/RPCResource.java 1970-01-01 00:00:00 +0000
+++ dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/rpc/RPCResource.java 2011-02-15 06:53:26 +0000
@@ -0,0 +1,189 @@
+package org.hisp.dhis.web.api.rpc;
+
+import java.net.URI;
+import java.util.List;
+import java.util.Set;
+
+import javax.ws.rs.Consumes;
+import javax.ws.rs.GET;
+import javax.ws.rs.POST;
+import javax.ws.rs.Path;
+import javax.ws.rs.PathParam;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.Context;
+import javax.ws.rs.core.MediaType;
+import javax.ws.rs.core.Response;
+import javax.ws.rs.core.Response.Status;
+import javax.ws.rs.core.UriInfo;
+
+import org.hisp.dhis.dataelement.DataElement;
+import org.hisp.dhis.dataelement.DataElementCategoryOptionCombo;
+import org.hisp.dhis.dataset.DataSet;
+import org.hisp.dhis.dataset.DataSetService;
+import org.hisp.dhis.datavalue.DataValue;
+import org.hisp.dhis.datavalue.DataValueService;
+import org.hisp.dhis.importexport.datavalueset.DataValueSet;
+import org.hisp.dhis.importexport.datavalueset.DataValueSetMapper;
+import org.hisp.dhis.organisationunit.OrganisationUnit;
+import org.springframework.beans.factory.annotation.Required;
+
+import com.ibatis.common.logging.Log;
+import com.ibatis.common.logging.LogFactory;
+
+@Path( "/rpc" )
+public class RPCResource
+{
+ private static Log log = LogFactory.getLog( RPCResource.class );
+
+ private DataValueSetMapper dataValueSetMapper;
+
+ private DataValueService dataValueService;
+
+ private DataSetService dataSetService;
+
+ @Context
+ UriInfo uriInfo;
+
+ @POST
+ @Path( "dataValueSets" )
+ @Consumes( MediaType.APPLICATION_XML )
+ public void storeDataValueSet( DataValueSet dataValueSet )
+ {
+ List<DataValue> dataValues = dataValueSetMapper.getDataValues( dataValueSet );
+
+ for ( DataValue dataValue : dataValues )
+ {
+ dataValueService.addDataValue( dataValue );
+ }
+ }
+
+ @GET
+ @Produces( MediaType.TEXT_HTML )
+ @Path( "dataSets" )
+ public String getDataValueSets()
+ {
+ return getDataValueSet();
+ }
+
+ @GET
+ @Produces( MediaType.TEXT_HTML )
+ public String getDataValueSet()
+ {
+ StringBuilder t = new StringBuilder();
+ t.append( head( "Data sets available for reporting" ) );
+
+ t.append( "<h2>Data sets available for reporting</h2>\n<ul>\n" );
+ for ( DataSet dataSet : dataSetService.getAllDataSets() )
+ {
+ URI uri = uriInfo.getBaseUriBuilder().path( "rpc/dataSets/{uuid}" ).build( dataSet.getUuid() );
+ t.append( "<li>" ).append( "<a href=\"" ).append( uri ).append( "\">" ).append( dataSet.getName() )
+ .append( "</a></li>\n" );
+ }
+ xmlTemplate( t );
+ t.append( tail() );
+
+ return t.toString();
+ }
+
+ @GET
+ @Path( "dataSets/{uuid}" )
+ @Produces( MediaType.TEXT_HTML )
+ public String getDataSet( @PathParam( "uuid" ) String uuid )
+ {
+
+ DataSet dataSet = dataSetService.getDataSet( uuid );
+
+ if ( dataSet == null )
+ {
+ throw new IllegalArgumentException( "No dataset with uuid " + uuid );
+ }
+
+ StringBuilder t = new StringBuilder();
+
+ t.append( head( "Data set " + dataSet.getName() ) );
+ t.append( "<p>Uuid: " ).append( dataSet.getUuid() ).append( "<br>\n" );
+ t.append( "Period type: " ).append( dataSet.getPeriodType().getName() ).append( " - " )
+ .append( dataSet.getPeriodType().getIsoFormat() );
+ t.append( "</p>\n" );
+
+ t.append( "<h2>Org units reporting data set</h2>\n<ul>" );
+ for ( OrganisationUnit unit : dataSet.getOrganisationUnis() )
+ {
+ t.append( "<li><b>" ).append( unit.getName() ).append( "</b> - " ).append( unit.getUuid() )
+ .append( "</li>" );
+ }
+ t.append( "</ul>\n" );
+
+ t.append( "<h2>Data elements in data set</h2>\n<ul>" );
+ for ( DataElement element : dataSet.getDataElements() )
+ {
+ t.append( "<li><b>" ).append( element.getName() ).append( "</b> (" ).append( element.getType() )
+ .append( ") - " ).append( element.getUuid() );
+
+ Set<DataElementCategoryOptionCombo> optionCombos = element.getCategoryCombo().getOptionCombos();
+ if ( optionCombos.size() > 1 )
+ {
+ t.append( "<br>CategoryOptionCombos\n<ul>\n" );
+ for ( DataElementCategoryOptionCombo optionCombo : optionCombos )
+ {
+ t.append( "<li><b>" ).append( optionCombo.getName() ).append( "</b> - " )
+ .append( optionCombo.getUuid() ).append( "</li>" );
+ }
+ t.append( "</ul>\n" );
+ }
+ t.append( "</li>\n" );
+ }
+ t.append( "</ul>" );
+ xmlTemplate( t );
+ t.append( tail() );
+
+ return t.toString();
+ }
+
+ private String head( String title )
+ {
+ return "<!DOCTYPE html PUBLIC \"-//W3C//DTD HTML 4.01//EN\" \"http://www.w3.org/TR/html4/strict.dtd\"> \n<html><head><title>"
+ + title + "</title></head>\n" + "<body>\n<h1>" + title + "</h1>\n";
+ }
+
+ private void xmlTemplate( StringBuilder t )
+ {
+ t.append( "<h2>Xml template</h2>\n" );
+
+ URI uri = uriInfo.getBaseUriBuilder().path( "rpc/dataValueSets" ).build();
+ t.append( "<p>Post according to the following template to " );
+ t.append( "<a href=\"" ).append( uri ).append( "\">" ).append( uri ).append( "</a>:</p>" );
+
+ t.append( "<pre>" ).append( "<dataValueSet xmlns=\"http://dhis2.org/schema/dataValueSet/0.1\"\n" );
+ t.append( " dataSet=\"dataSet UUID\" \n period=\"periodInIsoFormat\"\n orgUnit=\"unit UUID\"" );
+ t.append( "\n storedBy=\"user\">" );
+
+ t.append( "\n <dataValue dataElement=\"data element UUID\" categoryOptionCombo=\"UUID, only specify if used\" >value</dataValue>" );
+ t.append( "\n</dataValueSet>" );
+ t.append( "</pre>" );
+ }
+
+ private String tail()
+ {
+ return "</body>\n</html>\n";
+ }
+
+ @Required
+ public void setDataValueSetMapper( DataValueSetMapper dataValueSetMapper )
+ {
+ this.dataValueSetMapper = dataValueSetMapper;
+ }
+
+ @Required
+ public void setDataValueService( DataValueService dataValueService )
+ {
+ this.dataValueService = dataValueService;
+ }
+
+ @Required
+ public void setDataSetService( DataSetService dataSetService )
+ {
+ this.dataSetService = dataSetService;
+ }
+
+}

=== modified file 'dhis-2/dhis-web/dhis-web-api/src/main/resources/META-INF/dhis/beans.xml'
--- dhis-2/dhis-web/dhis-web-api/src/main/resources/META-INF/dhis/beans.xml 2011-01-21 18:28:26 +0000
+++ dhis-2/dhis-web/dhis-web-api/src/main/resources/META-INF/dhis/beans.xml 2011-02-15 06:53:26 +0000
@@ -60,6 +60,9 @@
  <bean id="org.hisp.dhis.web.api.mapping.NotAllowedExceptionMapper" class="org.hisp.dhis.web.api.mapping.NotAllowedExceptionMapper"
    scope="singleton" />

+ <bean id="org.hisp.dhis.web.api.mapping.IllegalArgumentExceptionMapper" class="org.hisp.dhis.web.api.mapping.IllegalArgumentExceptionMapper"
+ scope="singleton" />
+
  <bean id="JacksonJaxbJsonProvider" class="org.codehaus.jackson.jaxrs.JacksonJaxbJsonProvider" scope="singleton" />

  
@@ -76,6 +79,13 @@
  <bean id="org.hisp.dhis.web.api.service.ModelMapping" class="org.hisp.dhis.web.api.service.ModelMapping">
    <property name="categoryService" ref="org.hisp.dhis.dataelement.DataElementCategoryService" />
  </bean>
+
+
+ <bean id="org.hisp.dhis.web.api.rpc.RPCResource" class="org.hisp.dhis.web.api.rpc.RPCResource">
+ <property name="dataValueService" ref="org.hisp.dhis.datavalue.DataValueService" />
+ <property name="dataValueSetMapper" ref="org.hisp.dhis.importexport.datavalueset.DataValueSetMapper" />
+ <property name="dataSetService" ref="org.hisp.dhis.dataset.DataSetService" />
+ </bean>

</beans>

_______________________________________________
Mailing list: DHIS 2 developers in Launchpad
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : DHIS 2 developers in Launchpad
More help : ListHelp - Launchpad Help

Murod_Latifov · 15 February 2011 11:12

Hi Jo,

I am working on creating metaData object, which holds properties and relations of all objects in key/value format. In identifiable object I added metaCode Integer property. So each row in metaobject has unique key created for that metaobject, which in turn referenced by original object. This way we create uniqu sharable id for all metaobjects.

Also metaobjects and their relations could be easily populated from this metaobjects after fresh installation of DHIS2. So we have unique id (metaCode) accross installations for the same implementation (e.g. DataElement with different id in two different instances, can have same metaCode). While imp/exp we only need to reference metaCode, instead of object’s id. Looking at metaCode other party will now, what kind of object, and which particular instance is being processed.

Now I am working on table based metaObject, later this could be XMLized for other purposes like imp/exp validation agains metadata, or used as part of installation to populate metaobjects (DEs, DEgroups, DataSet, Sections, PeriodTypes, Indicators, OrgUnits).

You can see working version here:

bzr branch lp:~murodlatifov/+junk/dhis2

On each view for metaObjects, there is extra action icon for adding this object to meta table. For instance if DataSet to be added to meta table, all related DEs, Sections, OrgUnits are added before making DataSet a metaObject.

Now, I want to colorize each meta objects row to show if it is part of metadata, and so on.

regards,
murod

···

On Tue, Feb 15, 2011 at 11:14 AM, Jo Størset storset@gmail.com wrote:

Hi,

just commited a spike for simple(?) external posting of data values. This is a tricky area to design, so I’m hoping this can generate some discussions around it.

The immediate use case for this is an external server doing doing the data collection (mobile based), and then needing an easy way to post dataValueSets, with what I think is more detailed automated validation and feedback than the imp/exp would allow for…

You should be able to try it out by going to [1] (or correspondingly to your local instance) and follow the instructions on screen.

I’ve tried to promote dataSet and the corresponding dataValueSet as top level concepts, as I think this makes more sense than having isolated data values…

Also, I’ve created representation for dataValueSet, that is sort of dxf-like, but doesn’t really make for the batch-oriented import/export dxf/sdmx is intended for.

It is possible to tweak the format to be more batch-friendly, but I’m not convinced the use cases for this vs. batch are really similar enough that it makes sense at the cost of simplicity. One of the things this solution allows for, btw, is use of json instead of xml.

I think we do want an api more in this direction than the current import/export stuff will allow for, but I’m not in any way sure this is the right way to start off. In other words, maybe going with a layer above a reworked version of the xsl/xml based imp/exp framework would be better overall?

Anyway, have a quick look and feed back

Jo

[1] http://dhis.uio.no/dev/api/rpc

Den 15. feb. 2011 kl. 12.29 skrev noreply@launchpad.net:

revno: 2851

committer: Jo Størset storset@gmail.com

branch nick: dhis2

timestamp: Tue 2011-02-15 12:23:26 +0530

message:

Added spike for storing dataValueSets through a simple http post (see /api/rpc)

added:

dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/mapping/IllegalArgumentExceptionMapper.java

dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/rpc/

dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/rpc/RPCResource.java

modified:

dhis-2/dhis-web/dhis-web-api/src/main/resources/META-INF/dhis/beans.xml

–

lp:dhis2

https://code.launchpad.net/~dhis2-devs-core/dhis2/trunk

Your team DHIS 2 developers is subscribed to branch lp:dhis2.

To unsubscribe from this branch go to https://code.launchpad.net/~dhis2-devs-core/dhis2/trunk/+edit-subscription

=== added file ‘dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/mapping/IllegalArgumentExceptionMapper.java’

— dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/mapping/IllegalArgumentExceptionMapper.java 1970-01-01 00:00:00 +0000

+++ dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/mapping/IllegalArgumentExceptionMapper.java 2011-02-15 06:53:26 +0000

@@ -0,0 +1,23 @@

+package org.hisp.dhis.web.api.mapping;

+import javax.ws.rs.core.MediaType;

+import javax.ws.rs.core.Response;

+import javax.ws.rs.core.Response.Status;

+import javax.ws.rs.ext.ExceptionMapper;

+import javax.ws.rs.ext.Provider;

+import com.sun.jersey.spi.resource.Singleton;

+@Provider

+@Singleton

+public class IllegalArgumentExceptionMapper

implements ExceptionMapper

+{

@Override

public Response toResponse( IllegalArgumentException e )

{
   return Response.status( Status.CONFLICT ).entity( "Problem with input: " + e.getMessage() ).type( MediaType.TEXT_PLAIN ).build();
}

+}

=== added directory ‘dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/rpc’

=== added file ‘dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/rpc/RPCResource.java’

— dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/rpc/RPCResource.java 1970-01-01 00:00:00 +0000

+++ dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/rpc/RPCResource.java 2011-02-15 06:53:26 +0000

@@ -0,0 +1,189 @@

+package org.hisp.dhis.web.api.rpc;

+import java.net.URI;

+import java.util.List;

+import java.util.Set;

+import javax.ws.rs.Consumes;

+import javax.ws.rs.GET;

+import javax.ws.rs.POST;

+import javax.ws.rs.Path;

+import javax.ws.rs.PathParam;

+import javax.ws.rs.Produces;

+import javax.ws.rs.core.Context;

+import javax.ws.rs.core.MediaType;

+import javax.ws.rs.core.Response;

+import javax.ws.rs.core.Response.Status;

+import javax.ws.rs.core.UriInfo;

+import org.hisp.dhis.dataelement.DataElement;

+import org.hisp.dhis.dataelement.DataElementCategoryOptionCombo;

+import org.hisp.dhis.dataset.DataSet;

+import org.hisp.dhis.dataset.DataSetService;

+import org.hisp.dhis.datavalue.DataValue;

+import org.hisp.dhis.datavalue.DataValueService;

+import org.hisp.dhis.importexport.datavalueset.DataValueSet;

+import org.hisp.dhis.importexport.datavalueset.DataValueSetMapper;

+import org.hisp.dhis.organisationunit.OrganisationUnit;

+import org.springframework.beans.factory.annotation.Required;

+import com.ibatis.common.logging.Log;

+import com.ibatis.common.logging.LogFactory;

+@Path( “/rpc” )

+public class RPCResource

+{

private static Log log = LogFactory.getLog( RPCResource.class );

private DataValueSetMapper dataValueSetMapper;

private DataValueService dataValueService;

private DataSetService dataSetService;

@Context

UriInfo uriInfo;

@POST

@Path( “dataValueSets” )

@Consumes( MediaType.APPLICATION_XML )

public void storeDataValueSet( DataValueSet dataValueSet )

{
   List<DataValue> dataValues = dataValueSetMapper.getDataValues( dataValueSet );
   for ( DataValue dataValue : dataValues )
   {
       dataValueService.addDataValue( dataValue );
   }
}

@GET

@Produces( MediaType.TEXT_HTML )

@Path( “dataSets” )

public String getDataValueSets()

{
   return getDataValueSet();
}

@GET

@Produces( MediaType.TEXT_HTML )

public String getDataValueSet()

{
   StringBuilder t = new StringBuilder();
   t.append( head( "Data sets available for reporting" ) );
   t.append( "<h2>Data sets available for reporting</h2>\n<ul>\n" );
   for ( DataSet dataSet : dataSetService.getAllDataSets() )
   {
       URI uri = uriInfo.getBaseUriBuilder().path( "rpc/dataSets/{uuid}" ).build( dataSet.getUuid() );
       t.append( "<li>" ).append( "<a href=\"" ).append( uri ).append( "\">" ).append( dataSet.getName() )
           .append( "</a></li>\n" );
   }
   xmlTemplate( t );
   t.append( tail() );
   return t.toString();
}

@GET

@Path( “dataSets/{uuid}” )

@Produces( MediaType.TEXT_HTML )

public String getDataSet( @PathParam( “uuid” ) String uuid )

{
   DataSet dataSet = dataSetService.getDataSet( uuid );
   if ( dataSet == null )
   {
       throw new IllegalArgumentException( "No dataset with uuid " + uuid );
   }
   StringBuilder t = new StringBuilder();
   t.append( head( "Data set " + dataSet.getName() ) );
   t.append( "<p>Uuid: " ).append( dataSet.getUuid() ).append( "<br>\n" );
   t.append( "Period type: " ).append( dataSet.getPeriodType().getName() ).append( " - " )
       .append( dataSet.getPeriodType().getIsoFormat() );
   t.append( "</p>\n" );
   t.append( "<h2>Org units reporting data set</h2>\n<ul>" );
   for ( OrganisationUnit unit : dataSet.getOrganisationUnis() )
   {
       t.append( "<li><b>" ).append( unit.getName() ).append( "</b> - " ).append( unit.getUuid() )
           .append( "</li>" );
   }
   t.append( "</ul>\n" );
   t.append( "<h2>Data elements in data set</h2>\n<ul>" );
   for ( DataElement element : dataSet.getDataElements() )
   {
       t.append( "<li><b>" ).append( element.getName() ).append( "</b> (" ).append( element.getType() )
           .append( ") - " ).append( element.getUuid() );
       Set<DataElementCategoryOptionCombo> optionCombos = element.getCategoryCombo().getOptionCombos();
       if ( optionCombos.size() > 1 )
       {
           t.append( "<br>CategoryOptionCombos\n<ul>\n" );
           for ( DataElementCategoryOptionCombo optionCombo : optionCombos )
           {
               t.append( "<li><b>" ).append( optionCombo.getName() ).append( "</b> - " )
                   .append( optionCombo.getUuid() ).append( "</li>" );
           }
           t.append( "</ul>\n" );
       }
       t.append( "</li>\n" );
   }
   t.append( "</ul>" );
   xmlTemplate( t );
   t.append( tail() );
   return t.toString();
}

private String head( String title )

{
   return "<!DOCTYPE html PUBLIC \"-//W3C//DTD HTML 4.01//EN\" \"[http://www.w3.org/TR/html4/strict.dtd\](http://www.w3.org/TR/html4/strict.dtd%5C)"> \n<html><head><title>"
       + title + "</title></head>\n" + "<body>\n<h1>" + title + "</h1>\n";
}

private void xmlTemplate( StringBuilder t )

{
   t.append( "<h2>Xml template</h2>\n" );
   URI uri = uriInfo.getBaseUriBuilder().path( "rpc/dataValueSets" ).build();
   t.append( "<p>Post according to the following template to " );
   t.append( "<a href=\"" ).append( uri ).append( "\">" ).append( uri ).append( "</a>:</p>" );
   t.append( "<pre>" ).append( "&lt;dataValueSet xmlns=\"[http://dhis2.org/schema/dataValueSet/0.1\](http://dhis2.org/schema/dataValueSet/0.1%5C)"\n" );
   t.append( "    dataSet=\"dataSet UUID\" \n    period=\"periodInIsoFormat\"\n    orgUnit=\"unit UUID\"" );
   t.append( "\n    storedBy=\"user\"&gt;" );
   t.append( "\n  &lt;dataValue dataElement=\"data element UUID\" categoryOptionCombo=\"UUID, only specify if used\" &gt;value&lt;/dataValue&gt;" );
   t.append( "\n&lt;/dataValueSet&gt;" );
   t.append( "</pre>" );
}

private String tail()

{
   return "</body>\n</html>\n";
}

@Required

public void setDataValueSetMapper( DataValueSetMapper dataValueSetMapper )

{
   this.dataValueSetMapper = dataValueSetMapper;
}

@Required

public void setDataValueService( DataValueService dataValueService )

{
   this.dataValueService = dataValueService;
}

@Required

public void setDataSetService( DataSetService dataSetService )

{
   this.dataSetService = dataSetService;
}

+}

=== modified file ‘dhis-2/dhis-web/dhis-web-api/src/main/resources/META-INF/dhis/beans.xml’

— dhis-2/dhis-web/dhis-web-api/src/main/resources/META-INF/dhis/beans.xml 2011-01-21 18:28:26 +0000

+++ dhis-2/dhis-web/dhis-web-api/src/main/resources/META-INF/dhis/beans.xml 2011-02-15 06:53:26 +0000

@@ -60,6 +60,9 @@

<bean id=“org.hisp.dhis.web.api.mapping.NotAllowedExceptionMapper” class=“org.hisp.dhis.web.api.mapping.NotAllowedExceptionMapper”
scope="singleton" />
<bean id=“org.hisp.dhis.web.api.mapping.IllegalArgumentExceptionMapper” class=“org.hisp.dhis.web.api.mapping.IllegalArgumentExceptionMapper”

scope=“singleton” />

@@ -76,6 +79,13 @@
<property name="categoryService" ref="org.hisp.dhis.dataelement.DataElementCategoryService" />
Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Jo_Storset · 15 February 2011 13:01

Hi,

Forgot the little example using curl:

echo '<dataValueSet xmlns="http://dhis2.org/schema/dataValueSet/0.1" dataSet="9B93B662-9DFA-41C4-BEB5-9F3C352629DF" period="201102" orgUnit="A7D2ED55-D3B6-4CF0-ABC0-1AB2C8F547C6" storedBy="storset"><dataValue dataElement="C85D4EF1-4963-4F2D-B1FB-C76F2D6CCE03">111</dataValue> </dataValueSet>' | curl -v -HContent-Type:application/xml -u admin:district -d @- http://localhost:8080/api/rpc/dataValueSets/

(I would really like something a bit more meaningful than uuids, but thats another topic

Jo

bobj · 15 February 2011 13:10

Works very nicely. Just posted a file (dvset.xml):

using

curl http://admin:district@dhis.uio.no/dev/api/rpc/dataValueSets -H
"Content-Type: application/xml" -d @dvset.xml

Simple validation seems to work ok. I get an "Aw, Snap! ..." when
posting twice with the same period but that is probably something you
are not catching yet.

Hi,

just commited a spike for simple(?) external posting of data values. This is a tricky area to design, so I'm hoping this can generate some discussions around it.

The immediate use case for this is an external server doing doing the data collection (mobile based), and then needing an easy way to post dataValueSets, with what I think is more detailed automated validation and feedback than the imp/exp would allow for..

You should be able to try it out by going to [1] (or correspondingly to your local instance) and follow the instructions on screen.

I've tried to promote dataSet and the corresponding dataValueSet as top level concepts, as I think this makes more sense than having isolated data values..

Also, I've created representation for dataValueSet, that is sort of dxf-like, but doesn't really make for the batch-oriented import/export dxf/sdmx is intended for.
It is possible to tweak the format to be more batch-friendly, but I'm not convinced the use cases for this vs. batch are really similar enough that it makes sense at the cost of simplicity.

I don't agree with this. If we have to create two separate xml
dialects to handle what you term batch and non-batch (?) then we
really have things wrong. So we do need to harmonize this. Having
said that I am very happy to leave dxf v1 where it stands and look to
v2 which should be well enough designed to handle a variety of
scenarios..

Your use of DataValueSet here is very welcome - as you know I have
been advocating this for a while. Would be nice also to persist it to
provide audit (and simplify dtavalue store) but that is maybe too much
for now.

We would need to think about uuids - particularly with larger
payloads, but otherwise what you are doing here seems to be the
correct way to go. perhaps with the advent of stable identifiers, we
might have an optional uuid vs id tag to identify the items.

Looking good ....

Cheers
Bob

···

On 15 February 2011 10:14, Jo Størset <storset@gmail.com> wrote:

One of the things this solution allows for, btw, is use of json instead of xml.

I think we do want an api more in this direction than the current import/export stuff will allow for, but I'm not in any way sure this is the right way to start off. In other words, maybe going with a layer above a reworked version of the xsl/xml based imp/exp framework would be better overall?

Anyway, have a quick look and feed back
Jo

[1] http://dhis.uio.no/dev/api/rpc

Den 15. feb. 2011 kl. 12.29 skrev noreply@launchpad.net:

------------------------------------------------------------
revno: 2851
committer: Jo Størset <storset@gmail.com>
branch nick: dhis2
timestamp: Tue 2011-02-15 12:23:26 +0530
message:
Added spike for storing dataValueSets through a simple http post (see <dhis-root-url>/api/rpc)
added:
dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/mapping/IllegalArgumentExceptionMapper.java
dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/rpc/
dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/rpc/RPCResource.java
modified:
dhis-2/dhis-web/dhis-web-api/src/main/resources/META-INF/dhis/beans.xml

--
lp:dhis2
trunk : Code : DHIS

Your team DHIS 2 developers is subscribed to branch lp:dhis2.
To unsubscribe from this branch go to OpenID transaction in progress
=== added file 'dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/mapping/IllegalArgumentExceptionMapper.java'
--- dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/mapping/IllegalArgumentExceptionMapper.java 1970-01-01 00:00:00 +0000
+++ dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/mapping/IllegalArgumentExceptionMapper.java 2011-02-15 06:53:26 +0000
@@ -0,0 +1,23 @@
+package org.hisp.dhis.web.api.mapping;
+
+import javax.ws.rs.core.MediaType;
+import javax.ws.rs.core.Response;
+import javax.ws.rs.core.Response.Status;
+import javax.ws.rs.ext.ExceptionMapper;
+import javax.ws.rs.ext.Provider;
+
+import com.sun.jersey.spi.resource.Singleton;
+
+@Provider
+@Singleton
+public class IllegalArgumentExceptionMapper
+ implements ExceptionMapper<IllegalArgumentException>
+{
+
+ @Override
+ public Response toResponse( IllegalArgumentException e )
+ {
+ return Response.status( Status.CONFLICT ).entity( "Problem with input: " + e.getMessage() ).type( MediaType.TEXT_PLAIN ).build();
+ }
+
+}

=== added directory 'dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/rpc'
=== added file 'dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/rpc/RPCResource.java'
--- dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/rpc/RPCResource.java 1970-01-01 00:00:00 +0000
+++ dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/rpc/RPCResource.java 2011-02-15 06:53:26 +0000
@@ -0,0 +1,189 @@
+package org.hisp.dhis.web.api.rpc;
+
+import java.net.URI;
+import java.util.List;
+import java.util.Set;
+
+import javax.ws.rs.Consumes;
+import javax.ws.rs.GET;
+import javax.ws.rs.POST;
+import javax.ws.rs.Path;
+import javax.ws.rs.PathParam;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.Context;
+import javax.ws.rs.core.MediaType;
+import javax.ws.rs.core.Response;
+import javax.ws.rs.core.Response.Status;
+import javax.ws.rs.core.UriInfo;
+
+import org.hisp.dhis.dataelement.DataElement;
+import org.hisp.dhis.dataelement.DataElementCategoryOptionCombo;
+import org.hisp.dhis.dataset.DataSet;
+import org.hisp.dhis.dataset.DataSetService;
+import org.hisp.dhis.datavalue.DataValue;
+import org.hisp.dhis.datavalue.DataValueService;
+import org.hisp.dhis.importexport.datavalueset.DataValueSet;
+import org.hisp.dhis.importexport.datavalueset.DataValueSetMapper;
+import org.hisp.dhis.organisationunit.OrganisationUnit;
+import org.springframework.beans.factory.annotation.Required;
+
+import com.ibatis.common.logging.Log;
+import com.ibatis.common.logging.LogFactory;
+
+@Path( "/rpc" )
+public class RPCResource
+{
+ private static Log log = LogFactory.getLog( RPCResource.class );
+
+ private DataValueSetMapper dataValueSetMapper;
+
+ private DataValueService dataValueService;
+
+ private DataSetService dataSetService;
+
+ @Context
+ UriInfo uriInfo;
+
+ @POST
+ @Path( "dataValueSets" )
+ @Consumes( MediaType.APPLICATION_XML )
+ public void storeDataValueSet( DataValueSet dataValueSet )
+ {
+ List<DataValue> dataValues = dataValueSetMapper.getDataValues( dataValueSet );
+
+ for ( DataValue dataValue : dataValues )
+ {
+ dataValueService.addDataValue( dataValue );
+ }
+ }
+
+ @GET
+ @Produces( MediaType.TEXT_HTML )
+ @Path( "dataSets" )
+ public String getDataValueSets()
+ {
+ return getDataValueSet();
+ }
+
+ @GET
+ @Produces( MediaType.TEXT_HTML )
+ public String getDataValueSet()
+ {
+ StringBuilder t = new StringBuilder();
+ t.append( head( "Data sets available for reporting" ) );
+
+ t.append( "<h2>Data sets available for reporting</h2>\n<ul>\n" );
+ for ( DataSet dataSet : dataSetService.getAllDataSets() )
+ {
+ URI uri = uriInfo.getBaseUriBuilder().path( "rpc/dataSets/{uuid}" ).build( dataSet.getUuid() );
+ t.append( "<li>" ).append( "<a href=\"" ).append( uri ).append( "\">" ).append( dataSet.getName() )
+ .append( "</a></li>\n" );
+ }
+ xmlTemplate( t );
+ t.append( tail() );
+
+ return t.toString();
+ }
+
+ @GET
+ @Path( "dataSets/{uuid}" )
+ @Produces( MediaType.TEXT_HTML )
+ public String getDataSet( @PathParam( "uuid" ) String uuid )
+ {
+
+ DataSet dataSet = dataSetService.getDataSet( uuid );
+
+ if ( dataSet == null )
+ {
+ throw new IllegalArgumentException( "No dataset with uuid " + uuid );
+ }
+
+ StringBuilder t = new StringBuilder();
+
+ t.append( head( "Data set " + dataSet.getName() ) );
+ t.append( "<p>Uuid: " ).append( dataSet.getUuid() ).append( "<br>\n" );
+ t.append( "Period type: " ).append( dataSet.getPeriodType().getName() ).append( " - " )
+ .append( dataSet.getPeriodType().getIsoFormat() );
+ t.append( "</p>\n" );
+
+ t.append( "<h2>Org units reporting data set</h2>\n<ul>" );
+ for ( OrganisationUnit unit : dataSet.getOrganisationUnis() )
+ {
+ t.append( "<li><b>" ).append( unit.getName() ).append( "</b> - " ).append( unit.getUuid() )
+ .append( "</li>" );
+ }
+ t.append( "</ul>\n" );
+
+ t.append( "<h2>Data elements in data set</h2>\n<ul>" );
+ for ( DataElement element : dataSet.getDataElements() )
+ {
+ t.append( "<li><b>" ).append( element.getName() ).append( "</b> (" ).append( element.getType() )
+ .append( ") - " ).append( element.getUuid() );
+
+ Set<DataElementCategoryOptionCombo> optionCombos = element.getCategoryCombo().getOptionCombos();
+ if ( optionCombos.size() > 1 )
+ {
+ t.append( "<br>CategoryOptionCombos\n<ul>\n" );
+ for ( DataElementCategoryOptionCombo optionCombo : optionCombos )
+ {
+ t.append( "<li><b>" ).append( optionCombo.getName() ).append( "</b> - " )
+ .append( optionCombo.getUuid() ).append( "</li>" );
+ }
+ t.append( "</ul>\n" );
+ }
+ t.append( "</li>\n" );
+ }
+ t.append( "</ul>" );
+ xmlTemplate( t );
+ t.append( tail() );
+
+ return t.toString();
+ }
+
+ private String head( String title )
+ {
+ return "<!DOCTYPE html PUBLIC \"-//W3C//DTD HTML 4.01//EN\" \"http://www.w3.org/TR/html4/strict.dtd\"> \n<html><head><title>"
+ + title + "</title></head>\n" + "<body>\n<h1>" + title + "</h1>\n";
+ }
+
+ private void xmlTemplate( StringBuilder t )
+ {
+ t.append( "<h2>Xml template</h2>\n" );
+
+ URI uri = uriInfo.getBaseUriBuilder().path( "rpc/dataValueSets" ).build();
+ t.append( "<p>Post according to the following template to " );
+ t.append( "<a href=\"" ).append( uri ).append( "\">" ).append( uri ).append( "</a>:</p>" );
+
+ t.append( "<pre>" ).append( "<dataValueSet xmlns=\"http://dhis2.org/schema/dataValueSet/0.1\"\n" );
+ t.append( " dataSet=\"dataSet UUID\" \n period=\"periodInIsoFormat\"\n orgUnit=\"unit UUID\"" );
+ t.append( "\n storedBy=\"user\">" );
+
+ t.append( "\n <dataValue dataElement=\"data element UUID\" categoryOptionCombo=\"UUID, only specify if used\" >value</dataValue>" );
+ t.append( "\n</dataValueSet>" );
+ t.append( "</pre>" );
+ }
+
+ private String tail()
+ {
+ return "</body>\n</html>\n";
+ }
+
+ @Required
+ public void setDataValueSetMapper( DataValueSetMapper dataValueSetMapper )
+ {
+ this.dataValueSetMapper = dataValueSetMapper;
+ }
+
+ @Required
+ public void setDataValueService( DataValueService dataValueService )
+ {
+ this.dataValueService = dataValueService;
+ }
+
+ @Required
+ public void setDataSetService( DataSetService dataSetService )
+ {
+ this.dataSetService = dataSetService;
+ }
+
+}

=== modified file 'dhis-2/dhis-web/dhis-web-api/src/main/resources/META-INF/dhis/beans.xml'
--- dhis-2/dhis-web/dhis-web-api/src/main/resources/META-INF/dhis/beans.xml 2011-01-21 18:28:26 +0000
+++ dhis-2/dhis-web/dhis-web-api/src/main/resources/META-INF/dhis/beans.xml 2011-02-15 06:53:26 +0000
@@ -60,6 +60,9 @@
<bean id="org.hisp.dhis.web.api.mapping.NotAllowedExceptionMapper" class="org.hisp.dhis.web.api.mapping.NotAllowedExceptionMapper"
scope="singleton" />

+ <bean id="org.hisp.dhis.web.api.mapping.IllegalArgumentExceptionMapper" class="org.hisp.dhis.web.api.mapping.IllegalArgumentExceptionMapper"
+ scope="singleton" />
+
<bean id="JacksonJaxbJsonProvider" class="org.codehaus.jackson.jaxrs.JacksonJaxbJsonProvider" scope="singleton" />


@@ -76,6 +79,13 @@
<bean id="org.hisp.dhis.web.api.service.ModelMapping" class="org.hisp.dhis.web.api.service.ModelMapping">
<property name="categoryService" ref="org.hisp.dhis.dataelement.DataElementCategoryService" />
</bean>
+
+
+ <bean id="org.hisp.dhis.web.api.rpc.RPCResource" class="org.hisp.dhis.web.api.rpc.RPCResource">
+ <property name="dataValueService" ref="org.hisp.dhis.datavalue.DataValueService" />
+ <property name="dataValueSetMapper" ref="org.hisp.dhis.importexport.datavalueset.DataValueSetMapper" />
+ <property name="dataSetService" ref="org.hisp.dhis.dataset.DataSetService" />
+ </bean>

</beans>

_______________________________________________
Mailing list: DHIS 2 developers in Launchpad
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : DHIS 2 developers in Launchpad
More help : ListHelp - Launchpad Help

_______________________________________________
Mailing list: DHIS 2 developers in Launchpad
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : DHIS 2 developers in Launchpad
More help : ListHelp - Launchpad Help

bobj · 15 February 2011 13:11

Ah should have waited for the little example

···

On 15 February 2011 13:01, Jo Størset <storset@gmail.com> wrote:

Hi,

Forgot the little example using curl:

echo '<dataValueSet xmlns="http://dhis2.org/schema/dataValueSet/0.1" dataSet="9B93B662-9DFA-41C4-BEB5-9F3C352629DF" period="201102" orgUnit="A7D2ED55-D3B6-4CF0-ABC0-1AB2C8F547C6" storedBy="storset"><dataValue dataElement="C85D4EF1-4963-4F2D-B1FB-C76F2D6CCE03">111</dataValue> </dataValueSet>' | curl -v -HContent-Type:application/xml -u admin:district -d @- http://localhost:8080/api/rpc/dataValueSets/

(I would really like something a bit more meaningful than uuids, but thats another topic

Jo
_______________________________________________
Mailing list: DHIS 2 developers in Launchpad
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : DHIS 2 developers in Launchpad
More help : ListHelp - Launchpad Help

Jo_Storset · 15 February 2011 13:34

Simple validation seems to work ok. I get an "Aw, Snap! ..." when
posting twice with the same period but that is probably something you
are not catching yet.

Should work now.

I don't agree with this.

I know I don't necessarily agree myself, but it is also a matter of what is practically possible.. (And it might make sense to have a simpler json-oriented web api vs. a more fullfledged xml format for heavy imp/exp. Are you coming to Oslo in March by any chance, then we can fight it out! )

Your use of DataValueSet here is very welcome - as you know I have
been advocating this for a while. Would be nice also to persist it to
provide audit (and simplify dtavalue store) but that is maybe too much
for now.

Yes, that would have to be the next topic. Let's see if anyone else take the bait

Jo

···

Den 15. feb. 2011 kl. 18.40 skrev Bob Jolliffe:

Lars · 15 February 2011 14:09

Simple validation seems to work ok. I get an “Aw, Snap! …” when

posting twice with the same period but that is probably something you

are not catching yet.

Should work now.

I don’t agree with this.

I know I don’t necessarily agree myself, but it is also a matter of what is practically possible… (And it might make sense to have a simpler json-oriented web api vs. a more fullfledged xml format for heavy imp/exp. Are you coming to Oslo in March by any chance, then we can fight it out! )

Can you please explain why it is not practically possible to have dxf as the root element?

I don’t have anything against grouping datavalues in sets to make the format more compact. But, first, we currently don’t have any real requirements or use-case where we want to persist the “datavalueset”. Second we currently have no support for it in the model. So whats the point of modeling our exchange format this way? Yes we might need it sometime in the future but then we should implement it when we need it.

I also find it weird that we really need to implement two parsers for this. More work and more code to maintain.

The uuids will go for a new Identifier property for version 2.2 and make things less verbose btw.

Lars

···

On Tue, Feb 15, 2011 at 2:34 PM, Jo Størset storset@gmail.com wrote:

Den 15. feb. 2011 kl. 18.40 skrev Bob Jolliffe:

Your use of DataValueSet here is very welcome - as you know I have

been advocating this for a while. Would be nice also to persist it to

provide audit (and simplify dtavalue store) but that is maybe too much

for now.

Yes, that would have to be the next topic. Let’s see if anyone else take the bait

Jo

Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

bobj · 15 February 2011 14:11

Simple validation seems to work ok. I get an "Aw, Snap! ..." when
posting twice with the same period but that is probably something you
are not catching yet.

Should work now.

I don't agree with this.

I know I don't necessarily agree myself, but it is also a matter of what is practically possible.. (And it might make sense to have a simpler json-oriented web api vs. a more fullfledged xml format for heavy imp/exp. Are you coming to Oslo in March by any chance, then we can fight it out! )

28 March - 8 April

By heavy I presume you mean bulk loading of data, bootstrapping,
importing from legacy systems etc. For many of those scenarios csv is
maybe the best option anyway .. we do need to map out the use cases in
more detail than heavy vs mobile

What I don't want to do is to impose "heaviness" requirements on your
lighter use cases and I don't see why this should be necessary. For
example you have defined a lightweight element, datavalueset, which
meets your requirement for exchange. What I am suggesting is that we
should also define a compound element, dxf, which can allow
composition of these elements:

None of which prevents you using dataValueSet on its own. But we
really, really only want one place in dhis code where dataValue set is
marshalled/unmarshalled. You are now the jackson expert, but it looks
from what I can see that this does a good job in doing that and
certainly easier to maintain than staxwax.

One thought which was raised by Lars on looking at the MS OOXML format
used by, for example, xlsx, is that for xml elements which are likely
to be exchanged in bulk, we should perhaps look at using abbreviated
element/attribute names. So

becomes

Though, ironically, this starts to get closer to the json
representation (which might well be more suitable for bulk transfer).
Anyway .. still busy with other things but we will talk. Meanwhile
congrats on the "spike". This is the kind of api we need. But do try
to be a bit generous and think beyond your mobile use case

Cheers
Bob

···

On 15 February 2011 13:34, Jo Størset <storset@gmail.com> wrote:

Den 15. feb. 2011 kl. 18.40 skrev Bob Jolliffe:

Your use of DataValueSet here is very welcome - as you know I have
been advocating this for a while. Would be nice also to persist it to
provide audit (and simplify dtavalue store) but that is maybe too much
for now.

Yes, that would have to be the next topic. Let's see if anyone else take the bait

Jo

bobj · 15 February 2011 14:26

> Simple validation seems to work ok. I get an "Aw, Snap! ..." when
> posting twice with the same period but that is probably something you
> are not catching yet.

Should work now.

> I don't agree with this.

I know I don't necessarily agree myself, but it is also a matter of
what is practically possible.. (And it might make sense to have a simpler
json-oriented web api vs. a more fullfledged xml format for heavy imp/exp.
Are you coming to Oslo in March by any chance, then we can fight it out! )

Can you please explain why it is not practically possible to have dxf as the
root element?
I don't have anything against grouping datavalues in sets to make the format
more compact. But, first, we currently don't have any real requirements or
use-case where we want to persist the "datavalueset". Second we currently
have no support for it in the model. So whats the point of modeling our
exchange format this way?

Well partly because this structure models the way data is produced.
In sets. Off a form or off an import. SDMX data for example also
arrives in sets. While there is no support in the model it simply
means that we can lose information regarding the set. It becomes
important where you might want to rollback a set or identify where
some particular has come from. Currently this is sort of implicitly
keyed but there are benefits in making it explicit. For example you
can't currently trace a datavalue back to whether it was entered
through a dhis form, whether it arrived from one of Jo's 5000 phone's
or whether it was imported from iHRIS (or whatever). You can populate
the comment of all the datavalues but that's really expensive.

There are also savings to be had on storage by inheriting atttributes
like period and orrgunit from a dataset rather replicating in each
datavalue.

It's not a model change I would propose immediately (I think we have
enough zooks to sort out) but surely it is hard to argue that its not
the (proverbial) right thing to do. Meanwhile the way Jo has it in
his xml looks fine to me.

Cheers
Bob

···

On 15 February 2011 14:09, Lars Helge Øverland <larshelge@gmail.com> wrote:

On Tue, Feb 15, 2011 at 2:34 PM, Jo Størset <storset@gmail.com> wrote:

Den 15. feb. 2011 kl. 18.40 skrev Bob Jolliffe:

Yes we might need it sometime in the future but
then we should implement it when we need it.
I also find it weird that we really need to implement two parsers for this.
More work and more code to maintain.
The uuids will go for a new Identifier property for version 2.2 and make
things less verbose btw.
Lars

> Your use of DataValueSet here is very welcome - as you know I have
> been advocating this for a while. Would be nice also to persist it to
> provide audit (and simplify dtavalue store) but that is maybe too much
> for now.

Yes, that would have to be the next topic. Let's see if anyone else take
the bait

Jo
_______________________________________________
Mailing list: DHIS 2 developers in Launchpad
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : DHIS 2 developers in Launchpad
More help : ListHelp - Launchpad Help

Lars · 15 February 2011 14:37

Simple validation seems to work ok. I get an “Aw, Snap! …” when

posting twice with the same period but that is probably something you

are not catching yet.

Should work now.

I don’t agree with this.

I know I don’t necessarily agree myself, but it is also a matter of

what is practically possible… (And it might make sense to have a simpler

json-oriented web api vs. a more fullfledged xml format for heavy imp/exp.

Are you coming to Oslo in March by any chance, then we can fight it out! )

Can you please explain why it is not practically possible to have dxf as the

root element?

I don’t have anything against grouping datavalues in sets to make the format

more compact. But, first, we currently don’t have any real requirements or

use-case where we want to persist the “datavalueset”. Second we currently

have no support for it in the model. So whats the point of modeling our

exchange format this way?

Well partly because this structure models the way data is produced.

In sets. Off a form or off an import. SDMX data for example also

arrives in sets. While there is no support in the model it simply

means that we can lose information regarding the set. It becomes

important where you might want to rollback a set or identify where

some particular has come from. Currently this is sort of implicitly

keyed but there are benefits in making it explicit. For example you

can’t currently trace a datavalue back to whether it was entered

through a dhis form, whether it arrived from one of Jo’s 5000 phone’s

or whether it was imported from iHRIS (or whatever). You can populate

the comment of all the datavalues but that’s really expensive.

Yes thats fine, but my point is that we don’t support it in the model… So I still don’t understand why we should require senders to provide this info as it currently simply will be ignored.

There are also savings to be had on storage by inheriting atttributes

like period and orrgunit from a dataset rather replicating in each

datavalue.

That would require that we normalize and decompose into datavalueset (dataset,period,orgunit) and datavalue(dataelement, categorycomboid). And that would effectively change the whole system and not happen anytime soon…

···

On Tue, Feb 15, 2011 at 3:26 PM, Bob Jolliffe bobjolliffe@gmail.com wrote:

On 15 February 2011 14:09, Lars Helge Øverland larshelge@gmail.com wrote:

On Tue, Feb 15, 2011 at 2:34 PM, Jo Størset storset@gmail.com wrote:

Den 15. feb. 2011 kl. 18.40 skrev Bob Jolliffe:

It’s not a model change I would propose immediately (I think we have

enough zooks to sort out) but surely it is hard to argue that its not

the (proverbial) right thing to do. Meanwhile the way Jo has it in

his xml looks fine to me.

Cheers

Bob

Yes we might need it sometime in the future but

then we should implement it when we need it.

I also find it weird that we really need to implement two parsers for this.

More work and more code to maintain.

The uuids will go for a new Identifier property for version 2.2 and make

things less verbose btw.

Lars

Your use of DataValueSet here is very welcome - as you know I have

been advocating this for a while. Would be nice also to persist it to

provide audit (and simplify dtavalue store) but that is maybe too much

for now.

Yes, that would have to be the next topic. Let’s see if anyone else take

the bait

Jo

Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

bobj · 15 February 2011 15:13

>
>
>>
>>
>> > Simple validation seems to work ok. I get an "Aw, Snap! ..." when
>> > posting twice with the same period but that is probably something you
>> > are not catching yet.
>>
>> Should work now.
>>
>> > I don't agree with this.
>>
>> I know I don't necessarily agree myself, but it is also a matter of
>> what is practically possible.. (And it might make sense to have a
>> simpler
>> json-oriented web api vs. a more fullfledged xml format for heavy
>> imp/exp.
>> Are you coming to Oslo in March by any chance, then we can fight it
>> out! )
>>
>
> Can you please explain why it is not practically possible to have dxf as
> the
> root element?
> I don't have anything against grouping datavalues in sets to make the
> format
> more compact. But, first, we currently don't have any real requirements
> or
> use-case where we want to persist the "datavalueset". Second we
> currently
> have no support for it in the model. So whats the point of modeling our
> exchange format this way?

Well partly because this structure models the way data is produced.
In sets. Off a form or off an import. SDMX data for example also
arrives in sets. While there is no support in the model it simply
means that we can lose information regarding the set. It becomes
important where you might want to rollback a set or identify where
some particular has come from. Currently this is sort of implicitly
keyed but there are benefits in making it explicit. For example you
can't currently trace a datavalue back to whether it was entered
through a dhis form, whether it arrived from one of Jo's 5000 phone's
or whether it was imported from iHRIS (or whatever). You can populate
the comment of all the datavalues but that's really expensive.

Yes thats fine, but my point is that we don't support it in the model... So
I still don't understand why we should require senders to provide this info
as it currently simply will be ignored.

Well for a set of 100 datavalues you must either require them to
provide the attributes in Jo's datavalueset once or a 100 times
So it makes sense to use it. Even if we don't model and store the
datavalueset object itself.

There are also savings to be had on storage by inheriting atttributes
like period and orrgunit from a dataset rather replicating in each
datavalue.

That would require that we normalize and decompose into datavalueset
(dataset,period,orgunit) and datavalue(dataelement, categorycomboid). And
that would effectively change the whole system and not happen anytime
soon...

Agree. Mind you I suspect this would might prove to be a smaller
change to the whole system than the concept business which will be
more disruptive. The benefit of the encapsulation which we have in
the datavalue api is that period, source, storedBy etc are all
private so how the likes of getPeriod() works is easily reimplemented
and so much of the change would be transparent.

Anyway ... back to work

Cheers
Bob

···

On 15 February 2011 14:37, Lars Helge Øverland <larshelge@gmail.com> wrote:

On Tue, Feb 15, 2011 at 3:26 PM, Bob Jolliffe <bobjolliffe@gmail.com> wrote:

On 15 February 2011 14:09, Lars Helge Øverland <larshelge@gmail.com> >> wrote:
> On Tue, Feb 15, 2011 at 2:34 PM, Jo Størset <storset@gmail.com> wrote:
>> Den 15. feb. 2011 kl. 18.40 skrev Bob Jolliffe:

It's not a model change I would propose immediately (I think we have
enough zooks to sort out) but surely it is hard to argue that its not
the (proverbial) right thing to do. Meanwhile the way Jo has it in
his xml looks fine to me.

Cheers
Bob

> Yes we might need it sometime in the future but
> then we should implement it when we need it.
> I also find it weird that we really need to implement two parsers for
> this.
> More work and more code to maintain.
> The uuids will go for a new Identifier property for version 2.2 and make
> things less verbose btw.
> Lars
>
>>
>> > Your use of DataValueSet here is very welcome - as you know I have
>> > been advocating this for a while. Would be nice also to persist it
>> > to
>> > provide audit (and simplify dtavalue store) but that is maybe too
>> > much
>> > for now.
>>
>> Yes, that would have to be the next topic. Let's see if anyone else
>> take
>> the bait
>>
>> Jo
>> _______________________________________________
>> Mailing list: DHIS 2 developers in Launchpad
>> Post to : dhis2-devs@lists.launchpad.net
>> Unsubscribe : DHIS 2 developers in Launchpad
>> More help : ListHelp - Launchpad Help
>
>

Lars · 15 February 2011 15:16

Simple validation seems to work ok. I get an “Aw, Snap! …” when

posting twice with the same period but that is probably something you

are not catching yet.

Should work now.

I don’t agree with this.

I know I don’t necessarily agree myself, but it is also a matter of

what is practically possible… (And it might make sense to have a

simpler

json-oriented web api vs. a more fullfledged xml format for heavy

imp/exp.

Are you coming to Oslo in March by any chance, then we can fight it

out! )

Can you please explain why it is not practically possible to have dxf as

the

root element?

I don’t have anything against grouping datavalues in sets to make the

format

more compact. But, first, we currently don’t have any real requirements

or

use-case where we want to persist the “datavalueset”. Second we

currently

have no support for it in the model. So whats the point of modeling our

exchange format this way?

Well partly because this structure models the way data is produced.

In sets. Off a form or off an import. SDMX data for example also

arrives in sets. While there is no support in the model it simply

means that we can lose information regarding the set. It becomes

important where you might want to rollback a set or identify where

some particular has come from. Currently this is sort of implicitly

keyed but there are benefits in making it explicit. For example you

can’t currently trace a datavalue back to whether it was entered

through a dhis form, whether it arrived from one of Jo’s 5000 phone’s

or whether it was imported from iHRIS (or whatever). You can populate

the comment of all the datavalues but that’s really expensive.

Yes thats fine, but my point is that we don’t support it in the model… So

I still don’t understand why we should require senders to provide this info

as it currently simply will be ignored.

Well for a set of 100 datavalues you must either require them to

provide the attributes in Jo’s datavalueset once or a 100 times

So it makes sense to use it. Even if we don’t model and store the

datavalueset object itself.

Yes agree with that and I don’t have anything against grouping values for the benefit of making it compact. I am more concerned about having two different formats and parsers. We can discuss later.

···

On Tue, Feb 15, 2011 at 4:13 PM, Bob Jolliffe bobjolliffe@gmail.com wrote:

On 15 February 2011 14:37, Lars Helge Øverland larshelge@gmail.com wrote:

On Tue, Feb 15, 2011 at 3:26 PM, Bob Jolliffe bobjolliffe@gmail.com wrote:

On 15 February 2011 14:09, Lars Helge Øverland larshelge@gmail.com > > >> wrote:

On Tue, Feb 15, 2011 at 2:34 PM, Jo Størset storset@gmail.com wrote:

Den 15. feb. 2011 kl. 18.40 skrev Bob Jolliffe:

There are also savings to be had on storage by inheriting atttributes

like period and orrgunit from a dataset rather replicating in each

datavalue.

That would require that we normalize and decompose into datavalueset

(dataset,period,orgunit) and datavalue(dataelement, categorycomboid). And

that would effectively change the whole system and not happen anytime

soon…

Agree. Mind you I suspect this would might prove to be a smaller

change to the whole system than the concept business which will be

more disruptive. The benefit of the encapsulation which we have in

the datavalue api is that period, source, storedBy etc are all

private so how the likes of getPeriod() works is easily reimplemented

and so much of the change would be transparent.

Anyway … back to work

Cheers

Bob

It’s not a model change I would propose immediately (I think we have

enough zooks to sort out) but surely it is hard to argue that its not

the (proverbial) right thing to do. Meanwhile the way Jo has it in

his xml looks fine to me.

Cheers

Bob

Yes we might need it sometime in the future but

then we should implement it when we need it.

I also find it weird that we really need to implement two parsers for

this.

More work and more code to maintain.

The uuids will go for a new Identifier property for version 2.2 and make

things less verbose btw.

Lars

Your use of DataValueSet here is very welcome - as you know I have

been advocating this for a while. Would be nice also to persist it

to

provide audit (and simplify dtavalue store) but that is maybe too

much

for now.

Yes, that would have to be the next topic. Let’s see if anyone else

take

the bait

Jo

Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

bobj · 15 February 2011 15:22

>
>
>>
>> >
>> >
>> >>
>> >>
>> >> > Simple validation seems to work ok. I get an "Aw, Snap! ..." when
>> >> > posting twice with the same period but that is probably something
>> >> > you
>> >> > are not catching yet.
>> >>
>> >> Should work now.
>> >>
>> >> > I don't agree with this.
>> >>
>> >> I know I don't necessarily agree myself, but it is also a matter
>> >> of
>> >> what is practically possible.. (And it might make sense to have a
>> >> simpler
>> >> json-oriented web api vs. a more fullfledged xml format for heavy
>> >> imp/exp.
>> >> Are you coming to Oslo in March by any chance, then we can fight it
>> >> out! )
>> >>
>> >
>> > Can you please explain why it is not practically possible to have dxf
>> > as
>> > the
>> > root element?
>> > I don't have anything against grouping datavalues in sets to make the
>> > format
>> > more compact. But, first, we currently don't have any real
>> > requirements
>> > or
>> > use-case where we want to persist the "datavalueset". Second we
>> > currently
>> > have no support for it in the model. So whats the point of modeling
>> > our
>> > exchange format this way?
>>
>> Well partly because this structure models the way data is produced.
>> In sets. Off a form or off an import. SDMX data for example also
>> arrives in sets. While there is no support in the model it simply
>> means that we can lose information regarding the set. It becomes
>> important where you might want to rollback a set or identify where
>> some particular has come from. Currently this is sort of implicitly
>> keyed but there are benefits in making it explicit. For example you
>> can't currently trace a datavalue back to whether it was entered
>> through a dhis form, whether it arrived from one of Jo's 5000 phone's
>> or whether it was imported from iHRIS (or whatever). You can populate
>> the comment of all the datavalues but that's really expensive.
>
> Yes thats fine, but my point is that we don't support it in the model...
> So
> I still don't understand why we should require senders to provide this
> info
> as it currently simply will be ignored.

Well for a set of 100 datavalues you must either require them to
provide the attributes in Jo's datavalueset once or a 100 times
So it makes sense to use it. Even if we don't model and store the
datavalueset object itself.

Yes agree with that and I don't have anything against grouping values for
the benefit of making it compact. I am more concerned about having two
different formats and parsers.

Agree.

···

On 15 February 2011 15:16, Lars Helge Øverland <larshelge@gmail.com> wrote:

On Tue, Feb 15, 2011 at 4:13 PM, Bob Jolliffe <bobjolliffe@gmail.com> wrote:

On 15 February 2011 14:37, Lars Helge Øverland <larshelge@gmail.com> >> wrote:
> On Tue, Feb 15, 2011 at 3:26 PM, Bob Jolliffe <bobjolliffe@gmail.com> >> > wrote:
>> On 15 February 2011 14:09, Lars Helge Øverland <larshelge@gmail.com> >> >> wrote:
>> > On Tue, Feb 15, 2011 at 2:34 PM, Jo Størset <storset@gmail.com> >> >> > wrote:
>> >> Den 15. feb. 2011 kl. 18.40 skrev Bob Jolliffe:

We can discuss later.

>
>>
>> There are also savings to be had on storage by inheriting atttributes
>> like period and orrgunit from a dataset rather replicating in each
>> datavalue.
>
> That would require that we normalize and decompose into datavalueset
> (dataset,period,orgunit) and datavalue(dataelement, categorycomboid).
> And
> that would effectively change the whole system and not happen anytime
> soon...

Agree. Mind you I suspect this would might prove to be a smaller
change to the whole system than the concept business which will be
more disruptive. The benefit of the encapsulation which we have in
the datavalue api is that period, source, storedBy etc are all
private so how the likes of getPeriod() works is easily reimplemented
and so much of the change would be transparent.

Anyway ... back to work

Cheers
Bob

>
>>
>> It's not a model change I would propose immediately (I think we have
>> enough zooks to sort out) but surely it is hard to argue that its not
>> the (proverbial) right thing to do. Meanwhile the way Jo has it in
>> his xml looks fine to me.
>>
>> Cheers
>> Bob
>>
>> > Yes we might need it sometime in the future but
>> > then we should implement it when we need it.
>> > I also find it weird that we really need to implement two parsers for
>> > this.
>> > More work and more code to maintain.
>> > The uuids will go for a new Identifier property for version 2.2 and
>> > make
>> > things less verbose btw.
>> > Lars
>> >
>> >>
>> >> > Your use of DataValueSet here is very welcome - as you know I have
>> >> > been advocating this for a while. Would be nice also to persist
>> >> > it
>> >> > to
>> >> > provide audit (and simplify dtavalue store) but that is maybe too
>> >> > much
>> >> > for now.
>> >>
>> >> Yes, that would have to be the next topic. Let's see if anyone else
>> >> take
>> >> the bait
>> >>
>> >> Jo
>> >> _______________________________________________
>> >> Mailing list: DHIS 2 developers in Launchpad
>> >> Post to : dhis2-devs@lists.launchpad.net
>> >> Unsubscribe : DHIS 2 developers in Launchpad
>> >> More help : ListHelp - Launchpad Help
>> >
>> >
>
>

Murod_Latifov · 15 February 2011 15:27

Hi Bob,

Where are other possible dataelements and their categorycombos here? I don’t see this xml being valid considering how different dataSets will look like.

At least it should look something like this:

<dataValueSet xmlns=“http://dhis2.org/schema/dataValueSet/0.1”

dataSet="EEEE2762-6432-4C9A-

A739-8E7F1D74F60F"

period="2011"

orgUnit="59AC3C28-DDCE-435C-8695-8E457127DADC"

storedBy="Bob">

34

<...as many as they are>

etc. xml

cheers,
murod

···

On Tue, Feb 15, 2011 at 2:10 PM, Bob Jolliffe bobjolliffe@gmail.com wrote:

Works very nicely. Just posted a file (dvset.xml):

<dataValueSet xmlns=“http://dhis2.org/schema/dataValueSet/0.1”

dataSet=“EEEE2762-6432-4C9A-A739-8E7F1D74F60F”
period="2011"

orgUnit="59AC3C28-DDCE-435C-8695-8E457127DADC"

storedBy="Bob">
34

using

curl http://admin:district@dhis.uio.no/dev/api/rpc/dataValueSets -H

“Content-Type: application/xml” -d @dvset.xml

Simple validation seems to work ok. I get an “Aw, Snap! …” when

posting twice with the same period but that is probably something you

are not catching yet.

On 15 February 2011 10:14, Jo Størset storset@gmail.com wrote:

Hi,

just commited a spike for simple(?) external posting of data values. This is a tricky area to design, so I’m hoping this can generate some discussions around it.

The immediate use case for this is an external server doing doing the data collection (mobile based), and then needing an easy way to post dataValueSets, with what I think is more detailed automated validation and feedback than the imp/exp would allow for…

You should be able to try it out by going to [1] (or correspondingly to your local instance) and follow the instructions on screen.

I’ve tried to promote dataSet and the corresponding dataValueSet as top level concepts, as I think this makes more sense than having isolated data values…

Also, I’ve created representation for dataValueSet, that is sort of dxf-like, but doesn’t really make for the batch-oriented import/export dxf/sdmx is intended for.

It is possible to tweak the format to be more batch-friendly, but I’m not convinced the use cases for this vs. batch are really similar enough that it makes sense at the cost of simplicity.

I don’t agree with this. If we have to create two separate xml

dialects to handle what you term batch and non-batch (?) then we

really have things wrong. So we do need to harmonize this. Having

said that I am very happy to leave dxf v1 where it stands and look to

v2 which should be well enough designed to handle a variety of

scenarios…

Your use of DataValueSet here is very welcome - as you know I have

been advocating this for a while. Would be nice also to persist it to

provide audit (and simplify dtavalue store) but that is maybe too much

for now.

We would need to think about uuids - particularly with larger

payloads, but otherwise what you are doing here seems to be the

correct way to go. perhaps with the advent of stable identifiers, we

might have an optional uuid vs id tag to identify the items.

Looking good …

Cheers

Bob

One of the things this solution allows for, btw, is use of json instead of xml.

I think we do want an api more in this direction than the current import/export stuff will allow for, but I’m not in any way sure this is the right way to start off. In other words, maybe going with a layer above a reworked version of the xsl/xml based imp/exp framework would be better overall?

Anyway, have a quick look and feed back

Jo

[1] http://dhis.uio.no/dev/api/rpc

Den 15. feb. 2011 kl. 12.29 skrev noreply@launchpad.net:

revno: 2851

committer: Jo Størset storset@gmail.com

branch nick: dhis2

timestamp: Tue 2011-02-15 12:23:26 +0530

message:

Added spike for storing dataValueSets through a simple http post (see /api/rpc)

added:

dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/mapping/IllegalArgumentExceptionMapper.java

dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/rpc/

dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/rpc/RPCResource.java

modified:

dhis-2/dhis-web/dhis-web-api/src/main/resources/META-INF/dhis/beans.xml

–

lp:dhis2

https://code.launchpad.net/~dhis2-devs-core/dhis2/trunk

Your team DHIS 2 developers is subscribed to branch lp:dhis2.

To unsubscribe from this branch go to https://code.launchpad.net/~dhis2-devs-core/dhis2/trunk/+edit-subscription

=== added file ‘dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/mapping/IllegalArgumentExceptionMapper.java’

— dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/mapping/IllegalArgumentExceptionMapper.java 1970-01-01 00:00:00 +0000

+++ dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/mapping/IllegalArgumentExceptionMapper.java 2011-02-15 06:53:26 +0000

@@ -0,0 +1,23 @@

+package org.hisp.dhis.web.api.mapping;

+import javax.ws.rs.core.MediaType;

+import javax.ws.rs.core.Response;

+import javax.ws.rs.core.Response.Status;

+import javax.ws.rs.ext.ExceptionMapper;

+import javax.ws.rs.ext.Provider;

+import com.sun.jersey.spi.resource.Singleton;

+@Provider

+@Singleton

+public class IllegalArgumentExceptionMapper

implements ExceptionMapper

+{

@Override

public Response toResponse( IllegalArgumentException e )

{
   return Response.status( Status.CONFLICT ).entity( "Problem with input: " + e.getMessage() ).type( MediaType.TEXT_PLAIN ).build();
}

+}

=== added directory ‘dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/rpc’

=== added file ‘dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/rpc/RPCResource.java’

— dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/rpc/RPCResource.java 1970-01-01 00:00:00 +0000

+++ dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/rpc/RPCResource.java 2011-02-15 06:53:26 +0000

@@ -0,0 +1,189 @@

+package org.hisp.dhis.web.api.rpc;

+import java.net.URI;

+import java.util.List;

+import java.util.Set;

+import javax.ws.rs.Consumes;

+import javax.ws.rs.GET;

+import javax.ws.rs.POST;

+import javax.ws.rs.Path;

+import javax.ws.rs.PathParam;

+import javax.ws.rs.Produces;

+import javax.ws.rs.core.Context;

+import javax.ws.rs.core.MediaType;

+import javax.ws.rs.core.Response;

+import javax.ws.rs.core.Response.Status;

+import javax.ws.rs.core.UriInfo;

+import org.hisp.dhis.dataelement.DataElement;

+import org.hisp.dhis.dataelement.DataElementCategoryOptionCombo;

+import org.hisp.dhis.dataset.DataSet;

+import org.hisp.dhis.dataset.DataSetService;

+import org.hisp.dhis.datavalue.DataValue;

+import org.hisp.dhis.datavalue.DataValueService;

+import org.hisp.dhis.importexport.datavalueset.DataValueSet;

+import org.hisp.dhis.importexport.datavalueset.DataValueSetMapper;

+import org.hisp.dhis.organisationunit.OrganisationUnit;

+import org.springframework.beans.factory.annotation.Required;

+import com.ibatis.common.logging.Log;

+import com.ibatis.common.logging.LogFactory;

+@Path( “/rpc” )

+public class RPCResource

+{

private static Log log = LogFactory.getLog( RPCResource.class );

private DataValueSetMapper dataValueSetMapper;

private DataValueService dataValueService;

private DataSetService dataSetService;

@Context

UriInfo uriInfo;

@POST

@Path( “dataValueSets” )

@Consumes( MediaType.APPLICATION_XML )

public void storeDataValueSet( DataValueSet dataValueSet )

{
   List<DataValue> dataValues = dataValueSetMapper.getDataValues( dataValueSet );
   for ( DataValue dataValue : dataValues )
   {
       dataValueService.addDataValue( dataValue );
   }
}

@GET

@Produces( MediaType.TEXT_HTML )

@Path( “dataSets” )

public String getDataValueSets()

{
   return getDataValueSet();
}

@GET

@Produces( MediaType.TEXT_HTML )

public String getDataValueSet()

{
   StringBuilder t = new StringBuilder();
   t.append( head( "Data sets available for reporting" ) );
   t.append( "<h2>Data sets available for reporting</h2>\n<ul>\n" );
   for ( DataSet dataSet : dataSetService.getAllDataSets() )
   {
       URI uri = uriInfo.getBaseUriBuilder().path( "rpc/dataSets/{uuid}" ).build( dataSet.getUuid() );
       t.append( "<li>" ).append( "<a href=\"" ).append( uri ).append( "\">" ).append( dataSet.getName() )
           .append( "</a></li>\n" );
   }
   xmlTemplate( t );
   t.append( tail() );
   return t.toString();
}

@GET

@Path( “dataSets/{uuid}” )

@Produces( MediaType.TEXT_HTML )

public String getDataSet( @PathParam( “uuid” ) String uuid )

{
   DataSet dataSet = dataSetService.getDataSet( uuid );
   if ( dataSet == null )
   {
       throw new IllegalArgumentException( "No dataset with uuid " + uuid );
   }
   StringBuilder t = new StringBuilder();
   t.append( head( "Data set " + dataSet.getName() ) );
   t.append( "<p>Uuid: " ).append( dataSet.getUuid() ).append( "<br>\n" );
   t.append( "Period type: " ).append( dataSet.getPeriodType().getName() ).append( " - " )
       .append( dataSet.getPeriodType().getIsoFormat() );
   t.append( "</p>\n" );
   t.append( "<h2>Org units reporting data set</h2>\n<ul>" );
   for ( OrganisationUnit unit : dataSet.getOrganisationUnis() )
   {
       t.append( "<li><b>" ).append( unit.getName() ).append( "</b> - " ).append( unit.getUuid() )
           .append( "</li>" );
   }
   t.append( "</ul>\n" );
   t.append( "<h2>Data elements in data set</h2>\n<ul>" );
   for ( DataElement element : dataSet.getDataElements() )
   {
       t.append( "<li><b>" ).append( element.getName() ).append( "</b> (" ).append( element.getType() )
           .append( ") - " ).append( element.getUuid() );
       Set<DataElementCategoryOptionCombo> optionCombos = element.getCategoryCombo().getOptionCombos();
       if ( optionCombos.size() > 1 )
       {
           t.append( "<br>CategoryOptionCombos\n<ul>\n" );
           for ( DataElementCategoryOptionCombo optionCombo : optionCombos )
           {
               t.append( "<li><b>" ).append( optionCombo.getName() ).append( "</b> - " )
                   .append( optionCombo.getUuid() ).append( "</li>" );
           }
           t.append( "</ul>\n" );
       }
       t.append( "</li>\n" );
   }
   t.append( "</ul>" );
   xmlTemplate( t );
   t.append( tail() );
   return t.toString();
}

private String head( String title )

{
   return "<!DOCTYPE html PUBLIC \"-//W3C//DTD HTML 4.01//EN\" \"[http://www.w3.org/TR/html4/strict.dtd\](http://www.w3.org/TR/html4/strict.dtd%5C)"> \n<html><head><title>"
       + title + "</title></head>\n" + "<body>\n<h1>" + title + "</h1>\n";
}

private void xmlTemplate( StringBuilder t )

{
   t.append( "<h2>Xml template</h2>\n" );
   URI uri = uriInfo.getBaseUriBuilder().path( "rpc/dataValueSets" ).build();
   t.append( "<p>Post according to the following template to " );
   t.append( "<a href=\"" ).append( uri ).append( "\">" ).append( uri ).append( "</a>:</p>" );
   t.append( "<pre>" ).append( "&lt;dataValueSet xmlns=\"[http://dhis2.org/schema/dataValueSet/0.1\](http://dhis2.org/schema/dataValueSet/0.1%5C)"\n" );
   t.append( "    dataSet=\"dataSet UUID\" \n    period=\"periodInIsoFormat\"\n    orgUnit=\"unit UUID\"" );
   t.append( "\n    storedBy=\"user\"&gt;" );
   t.append( "\n  &lt;dataValue dataElement=\"data element UUID\" categoryOptionCombo=\"UUID, only specify if used\" &gt;value&lt;/dataValue&gt;" );
   t.append( "\n&lt;/dataValueSet&gt;" );
   t.append( "</pre>" );
}

private String tail()

{
   return "</body>\n</html>\n";
}

@Required

public void setDataValueSetMapper( DataValueSetMapper dataValueSetMapper )

{
   this.dataValueSetMapper = dataValueSetMapper;
}

@Required

public void setDataValueService( DataValueService dataValueService )

{
   this.dataValueService = dataValueService;
}

@Required

public void setDataSetService( DataSetService dataSetService )

{
   this.dataSetService = dataSetService;
}

+}

=== modified file ‘dhis-2/dhis-web/dhis-web-api/src/main/resources/META-INF/dhis/beans.xml’

— dhis-2/dhis-web/dhis-web-api/src/main/resources/META-INF/dhis/beans.xml 2011-01-21 18:28:26 +0000

+++ dhis-2/dhis-web/dhis-web-api/src/main/resources/META-INF/dhis/beans.xml 2011-02-15 06:53:26 +0000

@@ -60,6 +60,9 @@

<bean id=“org.hisp.dhis.web.api.mapping.NotAllowedExceptionMapper” class=“org.hisp.dhis.web.api.mapping.NotAllowedExceptionMapper”
scope="singleton" />
<bean id=“org.hisp.dhis.web.api.mapping.IllegalArgumentExceptionMapper” class=“org.hisp.dhis.web.api.mapping.IllegalArgumentExceptionMapper”

scope=“singleton” />

@@ -76,6 +79,13 @@
<property name="categoryService" ref="org.hisp.dhis.dataelement.DataElementCategoryService" />
Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Murod_Latifov · 15 February 2011 15:42

Single DataElement can have one or 100 categoryOptions, not counting how many DataElements Jo’s model has. And DataSet can have different DataElements of different Category Combos. That is how I see it.

···

2011/2/15 Bob Jolliffe bobjolliffe@gmail.com

On 15 February 2011 14:37, Lars Helge Øverland larshelge@gmail.com wrote:

On Tue, Feb 15, 2011 at 3:26 PM, Bob Jolliffe bobjolliffe@gmail.com wrote:

On 15 February 2011 14:09, Lars Helge Øverland larshelge@gmail.com > > >> wrote:

On Tue, Feb 15, 2011 at 2:34 PM, Jo Størset storset@gmail.com wrote:

Den 15. feb. 2011 kl. 18.40 skrev Bob Jolliffe:

Simple validation seems to work ok. I get an “Aw, Snap! …” when

posting twice with the same period but that is probably something you

are not catching yet.

Should work now.

I don’t agree with this.

I know I don’t necessarily agree myself, but it is also a matter of

what is practically possible… (And it might make sense to have a

simpler

json-oriented web api vs. a more fullfledged xml format for heavy

imp/exp.

Are you coming to Oslo in March by any chance, then we can fight it

out! )

Can you please explain why it is not practically possible to have dxf as

the

root element?

I don’t have anything against grouping datavalues in sets to make the

format

more compact. But, first, we currently don’t have any real requirements

or

use-case where we want to persist the “datavalueset”. Second we

currently

have no support for it in the model. So whats the point of modeling our

exchange format this way?

Well partly because this structure models the way data is produced.

In sets. Off a form or off an import. SDMX data for example also

arrives in sets. While there is no support in the model it simply

means that we can lose information regarding the set. It becomes

important where you might want to rollback a set or identify where

some particular has come from. Currently this is sort of implicitly

keyed but there are benefits in making it explicit. For example you

can’t currently trace a datavalue back to whether it was entered

through a dhis form, whether it arrived from one of Jo’s 5000 phone’s

or whether it was imported from iHRIS (or whatever). You can populate

the comment of all the datavalues but that’s really expensive.

Yes thats fine, but my point is that we don’t support it in the model… So

I still don’t understand why we should require senders to provide this info

as it currently simply will be ignored.

Well for a set of 100 datavalues you must either require them to

provide the attributes in Jo’s datavalueset once or a 100 times

So it makes sense to use it. Even if we don’t model and store the

datavalueset object itself.

There are also savings to be had on storage by inheriting atttributes

like period and orrgunit from a dataset rather replicating in each

datavalue.

That would require that we normalize and decompose into datavalueset

(dataset,period,orgunit) and datavalue(dataelement, categorycomboid). And

that would effectively change the whole system and not happen anytime

soon…

Agree. Mind you I suspect this would might prove to be a smaller

change to the whole system than the concept business which will be

more disruptive. The benefit of the encapsulation which we have in

the datavalue api is that period, source, storedBy etc are all

private so how the likes of getPeriod() works is easily reimplemented

and so much of the change would be transparent.

Anyway … back to work

Cheers

Bob

It’s not a model change I would propose immediately (I think we have

enough zooks to sort out) but surely it is hard to argue that its not

the (proverbial) right thing to do. Meanwhile the way Jo has it in

his xml looks fine to me.

Cheers

Bob

Yes we might need it sometime in the future but

then we should implement it when we need it.

I also find it weird that we really need to implement two parsers for

this.

More work and more code to maintain.

The uuids will go for a new Identifier property for version 2.2 and make

things less verbose btw.

Lars

Your use of DataValueSet here is very welcome - as you know I have

been advocating this for a while. Would be nice also to persist it

to

provide audit (and simplify dtavalue store) but that is maybe too

much

for now.

Yes, that would have to be the next topic. Let’s see if anyone else

take

the bait

Jo

Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

bobj · 15 February 2011 15:59

Hi Bob,

Where are other possible dataelements and their categorycombos here? I don't
see this xml being valid considering how different dataSets will look like.

Well trying with one datavalue was easier to type Whether a
datavalueset must necessarily be "complete" is a design issue. I
guess Jo could work it either way - but its nice not have to send lots
of empty elements if you've only got one value to send.

···

On 15 February 2011 15:27, Murod Latifov <mlatifov@gmail.com> wrote:

At least it should look something like this:

<dataValueSet xmlns="http://dhis2.org/schema/dataValueSet/0.1"
dataSet="EEEE2762-6432-4C9A-

A739-8E7F1D74F60F"
period="2011"
orgUnit="59AC3C28-DDCE-435C-8695-8E457127DADC"
storedBy="Bob">
<dataValue dataElements>

<dataValue>34
<dataElements>
<dataValue>
<dataElement uuid="uuid40">
<categoryCombo uuid="uuid40">
<value>34<value/>
<categoryCombo/>
<dataElement/>
<dataValue/>
<dataValue>
<dataElement uuid="uuid40">
<categoryCombo uuid="uuid40">
<value>34<value/>
<categoryCombo/>
<dataElement/>
<dataValue/>
<dataElements>
</dataValue>
<...as many as they are>

</dataValueSet>

etc. xml

cheers,
murod

On Tue, Feb 15, 2011 at 2:10 PM, Bob Jolliffe <bobjolliffe@gmail.com> wrote:

Works very nicely. Just posted a file (dvset.xml):

<dataValueSet xmlns="http://dhis2.org/schema/dataValueSet/0.1"
dataSet="EEEE2762-6432-4C9A-A739-8E7F1D74F60F"
period="2011"
orgUnit="59AC3C28-DDCE-435C-8695-8E457127DADC"
storedBy="Bob">
<dataValue dataElement="B01190B8-808C-42C4-AA6F-3F9CF51DC44F"
>34</dataValue>
</dataValueSet>

using

curl http://admin:district@dhis.uio.no/dev/api/rpc/dataValueSets -H
"Content-Type: application/xml" -d @dvset.xml

Simple validation seems to work ok. I get an "Aw, Snap! ..." when
posting twice with the same period but that is probably something you
are not catching yet.

On 15 February 2011 10:14, Jo Størset <storset@gmail.com> wrote:
> Hi,
>
> just commited a spike for simple(?) external posting of data values.
> This is a tricky area to design, so I'm hoping this can generate some
> discussions around it.
>
> The immediate use case for this is an external server doing doing the
> data collection (mobile based), and then needing an easy way to post
> dataValueSets, with what I think is more detailed automated validation and
> feedback than the imp/exp would allow for..
>
> You should be able to try it out by going to [1] (or correspondingly to
> your local instance) and follow the instructions on screen.
>
> I've tried to promote dataSet and the corresponding dataValueSet as top
> level concepts, as I think this makes more sense than having isolated data
> values..
>
> Also, I've created representation for dataValueSet, that is sort of
> dxf-like, but doesn't really make for the batch-oriented import/export
> dxf/sdmx is intended for.
> It is possible to tweak the format to be more batch-friendly, but I'm
> not convinced the use cases for this vs. batch are really similar enough
> that it makes sense at the cost of simplicity.

I don't agree with this. If we have to create two separate xml
dialects to handle what you term batch and non-batch (?) then we
really have things wrong. So we do need to harmonize this. Having
said that I am very happy to leave dxf v1 where it stands and look to
v2 which should be well enough designed to handle a variety of
scenarios..

Your use of DataValueSet here is very welcome - as you know I have
been advocating this for a while. Would be nice also to persist it to
provide audit (and simplify dtavalue store) but that is maybe too much
for now.

We would need to think about uuids - particularly with larger
payloads, but otherwise what you are doing here seems to be the
correct way to go. perhaps with the advent of stable identifiers, we
might have an optional uuid vs id tag to identify the items.

Looking good ....

Cheers
Bob

> One of the things this solution allows for, btw, is use of json instead
> of xml.
>
> I think we do want an api more in this direction than the current
> import/export stuff will allow for, but I'm not in any way sure this is the
> right way to start off. In other words, maybe going with a layer above a
> reworked version of the xsl/xml based imp/exp framework would be better
> overall?
>
> Anyway, have a quick look and feed back
> Jo
>
> [1] http://dhis.uio.no/dev/api/rpc
>
>
> Den 15. feb. 2011 kl. 12.29 skrev noreply@launchpad.net:
>
>> ------------------------------------------------------------
>> revno: 2851
>> committer: Jo Størset <storset@gmail.com>
>> branch nick: dhis2
>> timestamp: Tue 2011-02-15 12:23:26 +0530
>> message:
>> Added spike for storing dataValueSets through a simple http post (see
>> <dhis-root-url>/api/rpc)
>> added:
>>
>> dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/mapping/IllegalArgumentExceptionMapper.java
>> dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/rpc/
>>
>> dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/rpc/RPCResource.java
>> modified:
>>
>> dhis-2/dhis-web/dhis-web-api/src/main/resources/META-INF/dhis/beans.xml
>>
>>
>> --
>> lp:dhis2
>> trunk : Code : DHIS
>>
>> Your team DHIS 2 developers is subscribed to branch lp:dhis2.
>> To unsubscribe from this branch go to
>> OpenID transaction in progress
>> === added file
>> 'dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/mapping/IllegalArgumentExceptionMapper.java'
>> ---
>> dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/mapping/IllegalArgumentExceptionMapper.java
>> 1970-01-01 00:00:00 +0000
>> +++
>> dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/mapping/IllegalArgumentExceptionMapper.java
>> 2011-02-15 06:53:26 +0000
>> @@ -0,0 +1,23 @@
>> +package org.hisp.dhis.web.api.mapping;
>> +
>> +import javax.ws.rs.core.MediaType;
>> +import javax.ws.rs.core.Response;
>> +import javax.ws.rs.core.Response.Status;
>> +import javax.ws.rs.ext.ExceptionMapper;
>> +import javax.ws.rs.ext.Provider;
>> +
>> +import com.sun.jersey.spi.resource.Singleton;
>> +
>> +@Provider
>> +@Singleton
>> +public class IllegalArgumentExceptionMapper
>> + implements ExceptionMapper<IllegalArgumentException>
>> +{
>> +
>> + @Override
>> + public Response toResponse( IllegalArgumentException e )
>> + {
>> + return Response.status( Status.CONFLICT ).entity( "Problem
>> with input: " + e.getMessage() ).type( MediaType.TEXT_PLAIN ).build();
>> + }
>> +
>> +}
>>
>> === added directory
>> 'dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/rpc'
>> === added file
>> 'dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/rpc/RPCResource.java'
>> ---
>> dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/rpc/RPCResource.java
>> 1970-01-01 00:00:00 +0000
>> +++
>> dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/rpc/RPCResource.java
>> 2011-02-15 06:53:26 +0000
>> @@ -0,0 +1,189 @@
>> +package org.hisp.dhis.web.api.rpc;
>> +
>> +import java.net.URI;
>> +import java.util.List;
>> +import java.util.Set;
>> +
>> +import javax.ws.rs.Consumes;
>> +import javax.ws.rs.GET;
>> +import javax.ws.rs.POST;
>> +import javax.ws.rs.Path;
>> +import javax.ws.rs.PathParam;
>> +import javax.ws.rs.Produces;
>> +import javax.ws.rs.core.Context;
>> +import javax.ws.rs.core.MediaType;
>> +import javax.ws.rs.core.Response;
>> +import javax.ws.rs.core.Response.Status;
>> +import javax.ws.rs.core.UriInfo;
>> +
>> +import org.hisp.dhis.dataelement.DataElement;
>> +import org.hisp.dhis.dataelement.DataElementCategoryOptionCombo;
>> +import org.hisp.dhis.dataset.DataSet;
>> +import org.hisp.dhis.dataset.DataSetService;
>> +import org.hisp.dhis.datavalue.DataValue;
>> +import org.hisp.dhis.datavalue.DataValueService;
>> +import org.hisp.dhis.importexport.datavalueset.DataValueSet;
>> +import org.hisp.dhis.importexport.datavalueset.DataValueSetMapper;
>> +import org.hisp.dhis.organisationunit.OrganisationUnit;
>> +import org.springframework.beans.factory.annotation.Required;
>> +
>> +import com.ibatis.common.logging.Log;
>> +import com.ibatis.common.logging.LogFactory;
>> +
>> +@Path( "/rpc" )
>> +public class RPCResource
>> +{
>> + private static Log log = LogFactory.getLog( RPCResource.class );
>> +
>> + private DataValueSetMapper dataValueSetMapper;
>> +
>> + private DataValueService dataValueService;
>> +
>> + private DataSetService dataSetService;
>> +
>> + @Context
>> + UriInfo uriInfo;
>> +
>> + @POST
>> + @Path( "dataValueSets" )
>> + @Consumes( MediaType.APPLICATION_XML )
>> + public void storeDataValueSet( DataValueSet dataValueSet )
>> + {
>> + List<DataValue> dataValues = dataValueSetMapper.getDataValues(
>> dataValueSet );
>> +
>> + for ( DataValue dataValue : dataValues )
>> + {
>> + dataValueService.addDataValue( dataValue );
>> + }
>> + }
>> +
>> + @GET
>> + @Produces( MediaType.TEXT_HTML )
>> + @Path( "dataSets" )
>> + public String getDataValueSets()
>> + {
>> + return getDataValueSet();
>> + }
>> +
>> + @GET
>> + @Produces( MediaType.TEXT_HTML )
>> + public String getDataValueSet()
>> + {
>> + StringBuilder t = new StringBuilder();
>> + t.append( head( "Data sets available for reporting" ) );
>> +
>> + t.append( "<h2>Data sets available for reporting</h2>\n<ul>\n"
>> );
>> + for ( DataSet dataSet : dataSetService.getAllDataSets() )
>> + {
>> + URI uri = uriInfo.getBaseUriBuilder().path(
>> "rpc/dataSets/{uuid}" ).build( dataSet.getUuid() );
>> + t.append( "<li>" ).append( "<a href=\"" ).append( uri
>> ).append( "\">" ).append( dataSet.getName() )
>> + .append( "</a></li>\n" );
>> + }
>> + xmlTemplate( t );
>> + t.append( tail() );
>> +
>> + return t.toString();
>> + }
>> +
>> + @GET
>> + @Path( "dataSets/{uuid}" )
>> + @Produces( MediaType.TEXT_HTML )
>> + public String getDataSet( @PathParam( "uuid" ) String uuid )
>> + {
>> +
>> + DataSet dataSet = dataSetService.getDataSet( uuid );
>> +
>> + if ( dataSet == null )
>> + {
>> + throw new IllegalArgumentException( "No dataset with uuid
>> " + uuid );
>> + }
>> +
>> + StringBuilder t = new StringBuilder();
>> +
>> + t.append( head( "Data set " + dataSet.getName() ) );
>> + t.append( "<p>Uuid: " ).append( dataSet.getUuid() ).append(
>> "<br>\n" );
>> + t.append( "Period type: " ).append(
>> dataSet.getPeriodType().getName() ).append( " - " )
>> + .append( dataSet.getPeriodType().getIsoFormat() );
>> + t.append( "</p>\n" );
>> +
>> + t.append( "<h2>Org units reporting data set</h2>\n<ul>" );
>> + for ( OrganisationUnit unit : dataSet.getOrganisationUnis() )
>> + {
>> + t.append( "<li><b>" ).append( unit.getName() ).append(
>> "</b> - " ).append( unit.getUuid() )
>> + .append( "</li>" );
>> + }
>> + t.append( "</ul>\n" );
>> +
>> + t.append( "<h2>Data elements in data set</h2>\n<ul>" );
>> + for ( DataElement element : dataSet.getDataElements() )
>> + {
>> + t.append( "<li><b>" ).append( element.getName() ).append(
>> "</b> (" ).append( element.getType() )
>> + .append( ") - " ).append( element.getUuid() );
>> +
>> + Set<DataElementCategoryOptionCombo> optionCombos =
>> element.getCategoryCombo().getOptionCombos();
>> + if ( optionCombos.size() > 1 )
>> + {
>> + t.append( "<br>CategoryOptionCombos\n<ul>\n" );
>> + for ( DataElementCategoryOptionCombo optionCombo :
>> optionCombos )
>> + {
>> + t.append( "<li><b>" ).append(
>> optionCombo.getName() ).append( "</b> - " )
>> + .append( optionCombo.getUuid() ).append(
>> "</li>" );
>> + }
>> + t.append( "</ul>\n" );
>> + }
>> + t.append( "</li>\n" );
>> + }
>> + t.append( "</ul>" );
>> + xmlTemplate( t );
>> + t.append( tail() );
>> +
>> + return t.toString();
>> + }
>> +
>> + private String head( String title )
>> + {
>> + return "<!DOCTYPE html PUBLIC \"-//W3C//DTD HTML 4.01//EN\"
>> \"http://www.w3.org/TR/html4/strict.dtd\"> \n<html><head><title>"
>> + + title + "</title></head>\n" + "<body>\n<h1>" + title +
>> "</h1>\n";
>> + }
>> +
>> + private void xmlTemplate( StringBuilder t )
>> + {
>> + t.append( "<h2>Xml template</h2>\n" );
>> +
>> + URI uri = uriInfo.getBaseUriBuilder().path(
>> "rpc/dataValueSets" ).build();
>> + t.append( "<p>Post according to the following template to " );
>> + t.append( "<a href=\"" ).append( uri ).append( "\">" ).append(
>> uri ).append( "</a>:</p>" );
>> +
>> + t.append( "<pre>" ).append( "<dataValueSet
>> xmlns=\"http://dhis2.org/schema/dataValueSet/0.1\"\n" );
>> + t.append( " dataSet=\"dataSet UUID\" \n
>> period=\"periodInIsoFormat\"\n orgUnit=\"unit UUID\"" );
>> + t.append( "\n storedBy=\"user\">" );
>> +
>> + t.append( "\n <dataValue dataElement=\"data element UUID\"
>> categoryOptionCombo=\"UUID, only specify if used\"
>> >value</dataValue>" );
>> + t.append( "\n</dataValueSet>" );
>> + t.append( "</pre>" );
>> + }
>> +
>> + private String tail()
>> + {
>> + return "</body>\n</html>\n";
>> + }
>> +
>> + @Required
>> + public void setDataValueSetMapper( DataValueSetMapper
>> dataValueSetMapper )
>> + {
>> + this.dataValueSetMapper = dataValueSetMapper;
>> + }
>> +
>> + @Required
>> + public void setDataValueService( DataValueService dataValueService
>> )
>> + {
>> + this.dataValueService = dataValueService;
>> + }
>> +
>> + @Required
>> + public void setDataSetService( DataSetService dataSetService )
>> + {
>> + this.dataSetService = dataSetService;
>> + }
>> +
>> +}
>>
>> === modified file
>> 'dhis-2/dhis-web/dhis-web-api/src/main/resources/META-INF/dhis/beans.xml'
>> ---
>> dhis-2/dhis-web/dhis-web-api/src/main/resources/META-INF/dhis/beans.xml
>> 2011-01-21 18:28:26 +0000
>> +++
>> dhis-2/dhis-web/dhis-web-api/src/main/resources/META-INF/dhis/beans.xml
>> 2011-02-15 06:53:26 +0000
>> @@ -60,6 +60,9 @@
>> <bean id="org.hisp.dhis.web.api.mapping.NotAllowedExceptionMapper"
>> class="org.hisp.dhis.web.api.mapping.NotAllowedExceptionMapper"
>> scope="singleton" />
>>
>> + <bean
>> id="org.hisp.dhis.web.api.mapping.IllegalArgumentExceptionMapper"
>> class="org.hisp.dhis.web.api.mapping.IllegalArgumentExceptionMapper"
>> + scope="singleton" />
>> +
>> <bean id="JacksonJaxbJsonProvider"
>> class="org.codehaus.jackson.jaxrs.JacksonJaxbJsonProvider" scope="singleton"
>> />
>>
>> 
>> @@ -76,6 +79,13 @@
>> <bean id="org.hisp.dhis.web.api.service.ModelMapping"
>> class="org.hisp.dhis.web.api.service.ModelMapping">
>> <property name="categoryService"
>> ref="org.hisp.dhis.dataelement.DataElementCategoryService" />
>> </bean>
>> +
>> +
>> + <bean id="org.hisp.dhis.web.api.rpc.RPCResource"
>> class="org.hisp.dhis.web.api.rpc.RPCResource">
>> + <property name="dataValueService"
>> ref="org.hisp.dhis.datavalue.DataValueService" />
>> + <property name="dataValueSetMapper"
>> ref="org.hisp.dhis.importexport.datavalueset.DataValueSetMapper" />
>> + <property name="dataSetService"
>> ref="org.hisp.dhis.dataset.DataSetService" />
>> + </bean>
>>
>> </beans>
>>
>>
>> _______________________________________________
>> Mailing list: DHIS 2 developers in Launchpad
>> Post to : dhis2-devs@lists.launchpad.net
>> Unsubscribe : DHIS 2 developers in Launchpad
>> More help : ListHelp - Launchpad Help
>
>
> _______________________________________________
> Mailing list: DHIS 2 developers in Launchpad
> Post to : dhis2-devs@lists.launchpad.net
> Unsubscribe : DHIS 2 developers in Launchpad
> More help : ListHelp - Launchpad Help
>

_______________________________________________
Mailing list: DHIS 2 developers in Launchpad
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : DHIS 2 developers in Launchpad
More help : ListHelp - Launchpad Help

Jo_Storset · 16 February 2011 06:53

I guess it depends on the use case you are using the xml format for.. The xml itself is valid if it validates

My initial idea here was to try having (in http terms) POST instead of PUT semantics, a client would just have to post an update to the values it has, not having to send the complete set every time. I'm not sure it is a good solution, though.

Jo

···

Den 15. feb. 2011 kl. 20.57 skrev Murod Latifov:

Hi Bob,

Where are other possible dataelements and their categorycombos here? I don't see this xml being valid considering how different dataSets will look like.

Abyot_Gizaw · 16 February 2011 07:37

Simple validation seems to work ok. I get an “Aw, Snap! …” when
posting twice with the same period but that is probably something you

are not catching yet.

Should work now.

I don’t agree with this.

I know I don’t necessarily agree myself, but it is also a matter of

what is practically possible… (And it might make sense to have a simpler
json-oriented web api vs. a more fullfledged xml format for heavy imp/exp.
Are you coming to Oslo in March by any chance, then we can fight it out! )

Can you please explain why it is not practically possible to have dxf as the
root element?
I don’t have anything against grouping datavalues in sets to make the format
more compact. But, first, we currently don’t have any real requirements or

use-case where we want to persist the “datavalueset”. Second we currently
have no support for it in the model. So whats the point of modeling our
exchange format this way?

Well partly because this structure models the way data is produced.

In sets. Off a form or off an import. SDMX data for example also
arrives in sets. While there is no support in the model it simply
means that we can lose information regarding the set. It becomes
important where you might want to rollback a set or identify where

some particular has come from. Currently this is sort of implicitly
keyed but there are benefits in making it explicit. For example you
can’t currently trace a datavalue back to whether it was entered
through a dhis form, whether it arrived from one of Jo’s 5000 phone’s

or whether it was imported from iHRIS (or whatever). You can populate
the comment of all the datavalues but that’s really expensive.

There are also savings to be had on storage by inheriting atttributes

like period and orrgunit from a dataset rather replicating in each
datavalue.

It’s not a model change I would propose immediately (I think we have
enough zooks to sort out) but surely it is hard to argue that its not

the (proverbial) right thing to do. Meanwhile the way Jo has it in
his xml looks fine to me.

Hmmm… I think if we are to with this datavalueset concept and take away orgunit&period from datavalue and leave it with the groupset - we will be hitting a big trouble!

The flexibility we have right now - the way we define Indicators, design reports,… - is down to the independence of the datavalue. Each and every piece of datavalue can stand by itself and make sense - allowing monitoring and evaluation people greater flexibility and harmonization.

Again, with the datavalueset approach mentioned here, we will be against the minimum-dataset concept. For me the minimum dataset concept worked because users/healthprograms can share dataelement/datavalue.

Abyot.

···

2011/2/15 Bob Jolliffe bobjolliffe@gmail.com

On 15 February 2011 14:09, Lars Helge Øverland larshelge@gmail.com wrote:

On Tue, Feb 15, 2011 at 2:34 PM, Jo Størset storset@gmail.com wrote:

Den 15. feb. 2011 kl. 18.40 skrev Bob Jolliffe:

Cheers
Bob

Yes we might need it sometime in the future but
then we should implement it when we need it.
I also find it weird that we really need to implement two parsers for this.
More work and more code to maintain.

The uuids will go for a new Identifier property for version 2.2 and make
things less verbose btw.
Lars

Your use of DataValueSet here is very welcome - as you know I have

been advocating this for a while. Would be nice also to persist it to
provide audit (and simplify dtavalue store) but that is maybe too much
for now.

Yes, that would have to be the next topic. Let’s see if anyone else take

the bait

Jo

Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help : https://help.launchpad.net/ListHelp

bobj · 16 February 2011 10:57

>
>
>>
>>
>> > Simple validation seems to work ok. I get an "Aw, Snap! ..." when
>> > posting twice with the same period but that is probably something you
>> > are not catching yet.
>>
>> Should work now.
>>
>> > I don't agree with this.
>>
>> I know I don't necessarily agree myself, but it is also a matter of
>> what is practically possible.. (And it might make sense to have a
>> simpler
>> json-oriented web api vs. a more fullfledged xml format for heavy
>> imp/exp.
>> Are you coming to Oslo in March by any chance, then we can fight it
>> out! )
>>
>
> Can you please explain why it is not practically possible to have dxf as
> the
> root element?
> I don't have anything against grouping datavalues in sets to make the
> format
> more compact. But, first, we currently don't have any real requirements
> or
> use-case where we want to persist the "datavalueset". Second we
> currently
> have no support for it in the model. So whats the point of modeling our
> exchange format this way?

Well partly because this structure models the way data is produced.
In sets. Off a form or off an import. SDMX data for example also
arrives in sets. While there is no support in the model it simply
means that we can lose information regarding the set. It becomes
important where you might want to rollback a set or identify where
some particular has come from. Currently this is sort of implicitly
keyed but there are benefits in making it explicit. For example you
can't currently trace a datavalue back to whether it was entered
through a dhis form, whether it arrived from one of Jo's 5000 phone's
or whether it was imported from iHRIS (or whatever). You can populate
the comment of all the datavalues but that's really expensive.

There are also savings to be had on storage by inheriting atttributes
like period and orrgunit from a dataset rather replicating in each
datavalue.

It's not a model change I would propose immediately (I think we have
enough zooks to sort out) but surely it is hard to argue that its not
the (proverbial) right thing to do. Meanwhile the way Jo has it in
his xml looks fine to me.

Hmmm... I think if we are to with this datavalueset concept and take away
orgunit&period from datavalue and leave it with the groupset - we will be
hitting a big trouble!

The flexibility we have right now - the way we define Indicators, design
reports,... - is down to the independence of the datavalue. Each and every
piece of datavalue can stand by itself and make sense - allowing monitoring
and evaluation people greater flexibility and harmonization.

Again, with the datavalueset approach mentioned here, we will be against the
minimum-dataset concept. For me the minimum dataset concept worked because
users/healthprograms can share dataelement/datavalue.

I doubt the trouble would be as big as you think. But you might be
right and could be I'm missing something. But regarding just posting
of data it makes no difference at all other than making the message
more efficient.

What is minimum-dataset concept?

Bob

BTW its not really to do with groupset. But I guess that was a typo.

···

On 16 February 2011 07:37, Abyot Gizaw <abyota@gmail.com> wrote:

2011/2/15 Bob Jolliffe <bobjolliffe@gmail.com>

On 15 February 2011 14:09, Lars Helge Øverland <larshelge@gmail.com> >> wrote:
> On Tue, Feb 15, 2011 at 2:34 PM, Jo Størset <storset@gmail.com> wrote:
>> Den 15. feb. 2011 kl. 18.40 skrev Bob Jolliffe:

Abyot.

Cheers
Bob

> Yes we might need it sometime in the future but
> then we should implement it when we need it.
> I also find it weird that we really need to implement two parsers for
> this.
> More work and more code to maintain.
> The uuids will go for a new Identifier property for version 2.2 and make
> things less verbose btw.
> Lars
>
>>
>> > Your use of DataValueSet here is very welcome - as you know I have
>> > been advocating this for a while. Would be nice also to persist it
>> > to
>> > provide audit (and simplify dtavalue store) but that is maybe too
>> > much
>> > for now.
>>
>> Yes, that would have to be the next topic. Let's see if anyone else
>> take
>> the bait
>>
>> Jo
>> _______________________________________________
>> Mailing list: DHIS 2 developers in Launchpad
>> Post to : dhis2-devs@lists.launchpad.net
>> Unsubscribe : DHIS 2 developers in Launchpad
>> More help : ListHelp - Launchpad Help
>
>

_______________________________________________
Mailing list: DHIS 2 developers in Launchpad
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : DHIS 2 developers in Launchpad
More help : ListHelp - Launchpad Help