Export Metadata - Error with Content-Type header

Just my two cents on API versioning:

  • I agree that it’s good practice to retire API versions. Maintaining too many older versions is costly. The question is how many old versions to support.
  • API versioning should allow application developers to upgrade their DHIS2 server without requiring changes to their application, however the eventual goal should still be to migrate their application to use the newer API. The API versioning simply buys them time to do the migration.
  • Can we introduce deprecation warnings for previous API versions? i.e. If we plan to remove API xyz in v2.27, then in v2.25 and v2.26 we print deprecation warnings in the DHIS2 server log stating that API xyz will be removed in v2.27.
    Thoughts?

Cheers,

···

On Wed, Jun 15, 2016 at 12:48 PM, Lars Helge Øverland lars@dhis2.org wrote:

Hi Vanya,

retiring API versions is what most vendors do. There is a balance between the cost of maintaining old versions vs producing new features and providing a modern application. We know that things inevitably change over time due to needs from users.

The API versioning is meant to provide a comfortable upgrade path for app developers. We are trying to not break things, so ideally rolling over to a new version should introduce little changes on the client side.

You can e.g. see how google does it here:

https://developers.google.com/maps/documentation/javascript/versions#for-cutting-edge-applications

regards,

Lars


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

-doh

On Wed, Jun 15, 2016 at 7:23 AM, Morten Olav Hansen morten@dhis2.org wrote:

Well, there is already the case of metadata, where /api/metadata returns a different import report than /api/{23,24}/metadata, it will make those kind of changes easier (we have a few places where we wanted to clean up status codes etc, which we could not do as it would break apps)

To have true versioning we definitely need to introduce the DTO abstraction, but let’s see… not for 2.24 at least


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Lars Helge Øverland

Lead developer, DHIS 2

University of Oslo

Skype: larshelgeoverland

lars@dhis2.org

http://www.dhis2.org

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Wed, Jun 15, 2016 at 11:53 AM, Vanya Seth vanyas@thoughtworks.com wrote:

Adding DTO layer makes sense :). It will simplify stuff.

But unless you are looking at versioned DTO’s itself, I am not sure how long a way this might help you with versioning support.

Regards

Vanya

On Wed, Jun 15, 2016 at 9:04 AM, Morten Olav Hansen morten@dhis2.org wrote:

Hi Vanya

It’s not so much that we have to / want to remove older versions, it just that we can’t guarantee more than a few APi versions. Remember that we also are not using DTOs yet, so within a few versions, there are bound to be incompatibilities in the payloads. Discussions about DTOs are still going on.


With Regards
ThoughtWorks Technologies

Hyderabad

–Stay Hungry Stay Foolish!!

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Tue, Jun 14, 2016 at 7:48 PM, Vanya Seth vanyas@thoughtworks.com wrote:

Hi Morten

Supporting API versioning is great but why do we want to get rid of the previous versions as you explained in your email.

Well the whole point of supporting versioning is to let people continue to use the older versions without the fear of breaking anything.

I can understand that we might not support it (like bug fixing etc). But completely getting rid of it ??

Regards

Vanya

On Tue, Jun 14, 2016 at 2:58 PM, Paulo Grácio paulogracio@gmail.com wrote:

Ok, works fine with that version. Not sure if you guys are using Docker but if you are some images can be found here
https://hub.docker.com/r/pgracio/dhis2-web/tags/

Thanks,

Paulo

On Tue, Jun 14, 2016 at 10:55 AM Morten Olav Hansen morten@dhis2.org wrote:

Yes, that should be fine


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

With Regards
ThoughtWorks Technologies

Hyderabad

–Stay Hungry Stay Foolish!!

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Tue, Jun 14, 2016 at 3:36 PM, Paulo Grácio paulogracio@gmail.com wrote:

Ok, thanks for the clarification.

Can I use this war to build a docker image to run system tests?

https://www.dhis2.org/download/releases/trunk/dhis.war

BR,

Paulo

On Tue, Jun 14, 2016 at 10:33 AM Morten Olav Hansen morten@dhis2.org wrote:

Ah, ok… so in 2.23, only /api/23/metadata endpoint is versioned… nothing else, everything will be part of 2.24

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Tue, Jun 14, 2016 at 3:29 PM, Paulo Grácio paulogracio@gmail.com wrote:

Version:

2.23

Build revision:

22991

Build date:

2016-06-10 17:48

On Tue, Jun 14, 2016 at 10:20 AM Morten Olav Hansen morten@dhis2.org wrote:

  • In DHIS 2.24 will you keep the API version 23?

Yes, the plan is to keep 3 versions available, for 2.24 /api/23 and /api/24 is available, for 2.25 we will add /api/25, and for 2.26 we will add /api/26 and remove /api/23

Yes, what version are you on?

https://play.dhis2.org/dev/api/dataElements.json

https://play.dhis2.org/dev/api/23/dataElements.json

https://play.dhis2.org/dev/api/24/dataElements.json

All works on latest trunk

Morten

Thanks,

Paulo

On Tue, Jun 14, 2016 at 4:35 AM Morten Olav Hansen morten@dhis2.org wrote:

Hi Jason

Yes, the idea is that your app should target a specific API version. We are discussing letting /api/ always be the latest, -but- one of the reasons for adding versioning at all, was that we wanted to make more breaking changes… so if you app only targets /api/ and not /api/{version} there is a bigger risk that your app will break going forward.

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Tue, Jun 14, 2016 at 12:18 AM, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Morten,

I am a bit confused by this. Will we have to provide the explicit api version of the current version? So if we are on 2.24,will all API calls to the default API require the current version of the server?

Regards,

Jason

On Mon, Jun 13, 2016, 18:58 Paulo Grácio paulogracio@gmail.com wrote:

Ok, will create the tests based on /api/23/metadata.

Thanks,

Paulo

On Mon, Jun 13, 2016 at 11:17 AM Morten Olav Hansen morten@dhis2.org wrote:

No, that won’t work right now, that endpoint is anyways getting deprecated (replaced by /api/23/metadata and /api/24/metadata)


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Mon, Jun 13, 2016 at 4:11 PM, Paulo Grácio paulogracio@gmail.com wrote:

Hi Morten,

yes removing Content type header makes it work but shouldn’t it work with that header also? It works fine for other GET API calls.

/Paulo

On Mon, Jun 13, 2016 at 10:11 AM Morten Olav Hansen morten@dhis2.org wrote:

Hi Paulo

Is there any reason you are doing this request? I assume you want XML back? you dont need to set input content-type as you are not sending anything, and some of our internal gets a bit confused because of this… removing content-type makes it work

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Sun, Jun 12, 2016 at 10:47 PM, Paulo Grácio paulogracio@gmail.com wrote:

Hi all,

when calling the metadata API with Content-Type header it returns a 500.

curl -H “Accept: application/xml” -H “Content-Type: application/xml” -u admin:district “https://play.dhis2.org/demo/api/metadata?lastUpdated=2014-02

Best regards,

Paulo


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Hi David,

···

On Wed, Jun 15, 2016 at 9:45 AM, David Siang Fong Oh doh@thoughtworks.com wrote:

Just my two cents on API versioning:

  • I agree that it’s good practice to retire API versions. Maintaining too many older versions is costly. The question is how many old versions to support.
  • API versioning should allow application developers to upgrade their DHIS2 server without requiring changes to their application, however the eventual goal should still be to migrate their application to use the newer API. The API versioning simply buys them time to do the migration.
  • Can we introduce deprecation warnings for previous API versions? i.e. If we plan to remove API xyz in v2.27, then in v2.25 and v2.26 we print deprecation warnings in the DHIS2 server log stating that API xyz will be removed in v2.27.

Yes agreed. Sounds like a good idea to me.

regards,

Lars

Thoughts?

Cheers,

-doh

On Wed, Jun 15, 2016 at 12:48 PM, Lars Helge Øverland lars@dhis2.org wrote:

Hi Vanya,

retiring API versions is what most vendors do. There is a balance between the cost of maintaining old versions vs producing new features and providing a modern application. We know that things inevitably change over time due to needs from users.

The API versioning is meant to provide a comfortable upgrade path for app developers. We are trying to not break things, so ideally rolling over to a new version should introduce little changes on the client side.

You can e.g. see how google does it here:

https://developers.google.com/maps/documentation/javascript/versions#for-cutting-edge-applications

regards,

Lars


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

On Wed, Jun 15, 2016 at 7:23 AM, Morten Olav Hansen morten@dhis2.org wrote:

Well, there is already the case of metadata, where /api/metadata returns a different import report than /api/{23,24}/metadata, it will make those kind of changes easier (we have a few places where we wanted to clean up status codes etc, which we could not do as it would break apps)

To have true versioning we definitely need to introduce the DTO abstraction, but let’s see… not for 2.24 at least


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Lars Helge Øverland

Lead developer, DHIS 2

University of Oslo

Skype: larshelgeoverland

lars@dhis2.org

http://www.dhis2.org

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Wed, Jun 15, 2016 at 11:53 AM, Vanya Seth vanyas@thoughtworks.com wrote:

Adding DTO layer makes sense :). It will simplify stuff.

But unless you are looking at versioned DTO’s itself, I am not sure how long a way this might help you with versioning support.

Regards

Vanya

On Wed, Jun 15, 2016 at 9:04 AM, Morten Olav Hansen morten@dhis2.org wrote:

Hi Vanya

It’s not so much that we have to / want to remove older versions, it just that we can’t guarantee more than a few APi versions. Remember that we also are not using DTOs yet, so within a few versions, there are bound to be incompatibilities in the payloads. Discussions about DTOs are still going on.


With Regards
ThoughtWorks Technologies

Hyderabad

–Stay Hungry Stay Foolish!!

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Tue, Jun 14, 2016 at 7:48 PM, Vanya Seth vanyas@thoughtworks.com wrote:

Hi Morten

Supporting API versioning is great but why do we want to get rid of the previous versions as you explained in your email.

Well the whole point of supporting versioning is to let people continue to use the older versions without the fear of breaking anything.

I can understand that we might not support it (like bug fixing etc). But completely getting rid of it ??

Regards

Vanya

On Tue, Jun 14, 2016 at 2:58 PM, Paulo Grácio paulogracio@gmail.com wrote:

Ok, works fine with that version. Not sure if you guys are using Docker but if you are some images can be found here
https://hub.docker.com/r/pgracio/dhis2-web/tags/

Thanks,

Paulo

On Tue, Jun 14, 2016 at 10:55 AM Morten Olav Hansen morten@dhis2.org wrote:

Yes, that should be fine


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

With Regards
ThoughtWorks Technologies

Hyderabad

–Stay Hungry Stay Foolish!!

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Tue, Jun 14, 2016 at 3:36 PM, Paulo Grácio paulogracio@gmail.com wrote:

Ok, thanks for the clarification.

Can I use this war to build a docker image to run system tests?

https://www.dhis2.org/download/releases/trunk/dhis.war

BR,

Paulo

On Tue, Jun 14, 2016 at 10:33 AM Morten Olav Hansen morten@dhis2.org wrote:

Ah, ok… so in 2.23, only /api/23/metadata endpoint is versioned… nothing else, everything will be part of 2.24

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Tue, Jun 14, 2016 at 3:29 PM, Paulo Grácio paulogracio@gmail.com wrote:

Version:

2.23

Build revision:

22991

Build date:

2016-06-10 17:48

On Tue, Jun 14, 2016 at 10:20 AM Morten Olav Hansen morten@dhis2.org wrote:

  • In DHIS 2.24 will you keep the API version 23?

Yes, the plan is to keep 3 versions available, for 2.24 /api/23 and /api/24 is available, for 2.25 we will add /api/25, and for 2.26 we will add /api/26 and remove /api/23

Yes, what version are you on?

https://play.dhis2.org/dev/api/dataElements.json

https://play.dhis2.org/dev/api/23/dataElements.json

https://play.dhis2.org/dev/api/24/dataElements.json

All works on latest trunk

Morten

Thanks,

Paulo

On Tue, Jun 14, 2016 at 4:35 AM Morten Olav Hansen morten@dhis2.org wrote:

Hi Jason

Yes, the idea is that your app should target a specific API version. We are discussing letting /api/ always be the latest, -but- one of the reasons for adding versioning at all, was that we wanted to make more breaking changes… so if you app only targets /api/ and not /api/{version} there is a bigger risk that your app will break going forward.

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Tue, Jun 14, 2016 at 12:18 AM, Jason Pickering jason.p.pickering@gmail.com wrote:

Hi Morten,

I am a bit confused by this. Will we have to provide the explicit api version of the current version? So if we are on 2.24,will all API calls to the default API require the current version of the server?

Regards,

Jason

On Mon, Jun 13, 2016, 18:58 Paulo Grácio paulogracio@gmail.com wrote:

Ok, will create the tests based on /api/23/metadata.

Thanks,

Paulo

On Mon, Jun 13, 2016 at 11:17 AM Morten Olav Hansen morten@dhis2.org wrote:

No, that won’t work right now, that endpoint is anyways getting deprecated (replaced by /api/23/metadata and /api/24/metadata)


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Mon, Jun 13, 2016 at 4:11 PM, Paulo Grácio paulogracio@gmail.com wrote:

Hi Morten,

yes removing Content type header makes it work but shouldn’t it work with that header also? It works fine for other GET API calls.

/Paulo

On Mon, Jun 13, 2016 at 10:11 AM Morten Olav Hansen morten@dhis2.org wrote:

Hi Paulo

Is there any reason you are doing this request? I assume you want XML back? you dont need to set input content-type as you are not sending anything, and some of our internal gets a bit confused because of this… removing content-type makes it work

Morten Olav Hansen

Senior Engineer, DHIS 2

University of Oslo

http://www.dhis2.org

On Sun, Jun 12, 2016 at 10:47 PM, Paulo Grácio paulogracio@gmail.com wrote:

Hi all,

when calling the metadata API with Content-Type header it returns a 500.

curl -H “Accept: application/xml” -H “Content-Type: application/xml” -u admin:district “https://play.dhis2.org/demo/api/metadata?lastUpdated=2014-02

Best regards,

Paulo


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

Lars Helge Øverland

Lead developer, DHIS 2

University of Oslo

Skype: larshelgeoverland

lars@dhis2.org

http://www.dhis2.org