Dear Uwe,
This is a json issue. Recently i had guidance from lars that the xml returns even the empty properties.
Apparently, suggestions in the link below talks of leaving out properties with null values.
https://google.github.io/styleguide/jsoncstyleguide.xml
The only work around:
1. Try using xml format ( if ok with U)
2. You will need to take back missing properties within your codes ( probably your previous work around)
On Dec 16, 2016 15:33, "Uwe Wahser" <uwe@wahser.de > <mailto:uwe@wahser.de>> wrote:
Hi Jason,
It's not new - I had that problem already last year, but managed
to work around. Now I wanted to use more properties and the
workaround doesn't work for that.
I had decided against csv back then, but forgot why. Must have
been a good reason, because normally I prefer csv 
I also think that it is not a DHIS issue and that the kettle
converter should be able to do that. But DHIS developers are
usually more responsive
I was hoping that it wouldn't be a big deal, as it works with csv.
Implementing it as a parameter would also give the choice of using
that feature if needed, thus avoiding inflated payloads per default.
Anyway, I think I'll enjoy our Xmas party now. Maybe I that gives
me better ideas on Monday 
Thanks for looking at it,
Uwe
Sent with AquaMail for Android
http://www.aqua-mail.com
On 16 December 2016 12:54:17 Jason Pickering > <jason.p.pickering@gmail.com <mailto:jason.p.pickering@gmail.com>> > wrote:
Hi Uwe,
OK, I sort of see the problem you have, but am pretty sure its
not a DHIS2 problem. Are you saying that this is new behaviour,
and it was not like this before? I just checked an old instance
I have which is running 2.17, and it is exactly the same, so not
sure this has changed any time in the recent past.
Maybe you should try and use CSV?
https://play.dhis2.org/demo/api/organisationUnits.csv?paging=false&filter=level:eq:3&fields=name,comment
<https://play.dhis2.org/demo/api/organisationUnits.csv?paging=false&filter=level:eq:3&fields=name,comment>
does produce an extra column (with NULLS).
From the R standpoint (as well as other languages I think), its
really up to you to parse the payload as you need it, but its
handled quite easily for this case in R. Other times, the nested
lists need to be flattened, if that is what you require, but I do
not see this as DHIS2 issue really. It would make little sense
to include all of these NULLs where they do not exist, as it
could drastically increase the size of the payload. That is why
they are not included, except where not null.
Inline image 1
On Fri, Dec 16, 2016 at 10:41 AM, Uwe Wahser <uwe@wahser.de >> <mailto:uwe@wahser.de>> wrote:
Hi Jason,
simple example:
https://play.dhis2.org/demo/api/organisationUnits.json?paging=false&filter=level:eq:3&fields=name,comment
<https://play.dhis2.org/demo/api/organisationUnits.json?paging=false&filter=level:eq:3&fields=name,comment>
What I'd expect would be a json package with name & comment
for ALL organisationUnits, even when the comment is NULL.
What I get is a comment property for Badjia (which I created
for the example), for all the other I only get the name property:
[{"name":"Badjia","comment":"This is a
comment."},{"name":"Bagruwa"},{"name":"Baoma"}...]
Converting this package in kettle would result to an error
message that 50 (or whatever) instances of name were found,
but only 1 instance of comment. kettle expects always the
complete set of properties for all objects. Not sure how R or
others handle this ...
If there was a query parameter like returnNull=true , that
could give back the null properties like
{"name":"Bagruwa","comment":""}, it would be possible to work
with these packages. Actually I had expected this to be the
default behaviour, when specifying fields.
Best regards,
Uwe
---
> Jason Pickering <jason.p.pickering@gmail.com
<mailto:jason.p.pickering@gmail.com>> hat am 16. Dezember
2016 um 12:05 geschrieben:
>
> Hi Uwe,
> Could you give a specific example from the demo server with
an API call,
> and what is not working as you expect?
>
> Regards,
> Jason
>
> On Fri, Dec 16, 2016 at 8:48 AM, Uwe Wahser <uwe@wahser.de >> <mailto:uwe@wahser.de>> wrote:
>
> > Sorry, on a closer look it only seems to work for empty
collections.
> > Properties of the object itself (e.g. email, comments of
organizationUnits)
> > are still left out.
> >
> > The floor is open again for other suggestions
> >
> > Regards, Uwe
> >
> > > Uwe Wahser <uwe@wahser.de <mailto:uwe@wahser.de>> hat
am 16. Dezember 2016 um 10:24 >> > > geschrieben:
> > >
> > > Thanks a lot, Jason, I will try that. I tried 'fields='
expecting this
> > would be the same as 'fields=:all' , which is obviously
wrong.
> > >
> > > That's a very useful information - do you mind
including it into the
> > manual?
> > >
> > > Regards, Uwe
> > >
> > > ---
> > >
> > > > Jason Pickering <jason.p.pickering@gmail.com
<mailto:jason.p.pickering@gmail.com>> hat am 16. Dezember
> > 2016 um 10:13 geschrieben:
> > > >
> > > > Hi Uwe
> > > > Have a look here
> > > >
> > > > https://ci.dhis2.org/docs/master/en/developer/html/
<https://ci.dhis2.org/docs/master/en/developer/html/>
> > webapi_metadata_field_filter.html
> > > >
> > > > and specifically the use of the ?fields=:all paramater.
> > > >
> > > > Regards,
> > > > Jason
> > > >
> > > > On Fri, Dec 16, 2016 at 7:58 AM, Uwe Wahser >> <uwe@wahser.de <mailto:uwe@wahser.de>> wrote:
> > > >
> > > > > Dear all,
> > > > >
> > > > > I am having a bit of an issue when querying
metadata from the api in
> > json
> > > > > format and further processing them in kettle
(Pentaho DI): DHIS2 only
> > > > > returns those properties in the json-package that
are not NULL.
> > However,
> > > > > the json converter in kettle needs the full
property-set in order to
> > create
> > > > > a tabular structure from the data (don't know the
reason for that).
> > > > >
> > > > > Is there a way to force the api to return the full
property set,
> > e.g. a
> > > > > secret flag? I'd actually expect to get the full
list - even if the
> > > > > property is NULL -, at least when I specify a list
of properties in
> > the
> > > > > field-filter ...
> > > > >
> > > > > Of course, I could also ask the kettle-guys to be a
bit more
> > flexible, but
> > > > > normally they are not as agile as the DHIS2-DEVs 
> > > > >
> > > > > Best regards,
> > > > >
> > > > > Uwe
> > > > >
> > > > > _______________________________________________
> > > > > Mailing list: https://launchpad.net/~dhis2-users
<https://launchpad.net/~dhis2-users>
> > > > > Post to : dhis2-users@lists.launchpad.net
<mailto:dhis2-users@lists.launchpad.net>
> > > > > Unsubscribe : https://launchpad.net/~dhis2-users
<https://launchpad.net/~dhis2-users>
> > > > > More help : https://help.launchpad.net/ListHelp
<https://help.launchpad.net/ListHelp>
> > > > >
> > > >
> > > > --
> > > > Jason P. Pickering
> > > > email: jason.p.pickering@gmail.com
<mailto:jason.p.pickering@gmail.com>
> > > > tel:+46764147049 <tel:%2B46764147049>
> > >
> > > _______________________________________________
> > > Mailing list: https://launchpad.net/~dhis2-users
<https://launchpad.net/~dhis2-users>
> > > Post to : dhis2-users@lists.launchpad.net
<mailto:dhis2-users@lists.launchpad.net>
> > > Unsubscribe : https://launchpad.net/~dhis2-users
<https://launchpad.net/~dhis2-users>
> > > More help : https://help.launchpad.net/ListHelp
<https://help.launchpad.net/ListHelp>
> >
>
> --
> Jason P. Pickering
> email: jason.p.pickering@gmail.com
<mailto:jason.p.pickering@gmail.com>
> tel:+46764147049 <tel:%2B46764147049>
-- Jason P. Pickering
email: jason.p.pickering@gmail.com
<mailto:jason.p.pickering@gmail.com>
tel:+46764147049 <tel:+46%2076%20414%2070%2049>
_______________________________________________
Mailing list: https://launchpad.net/~dhis2-users
<https://launchpad.net/~dhis2-users>
Post to : dhis2-users@lists.launchpad.net
<mailto:dhis2-users@lists.launchpad.net>
Unsubscribe : https://launchpad.net/~dhis2-users
<https://launchpad.net/~dhis2-users>
More help : https://help.launchpad.net/ListHelp
<https://help.launchpad.net/ListHelp>
