[Bug 1617416] [NEW] LFs removed on import, breaking custom JS

Timothy_Harding1 · 26 August 2016 18:05

Public bug reported:

Hello Devs

It looks like uploads against both /metaData and /xx/metadata are
removing LFs in their entirety from <dataEntryForms> htmlcode. I
prepared a table to show:

···

+---------+-----------------------+---------------------------------+

Version | Upload Method | Result |

+---------+-----------------------+---------------------------------+

   2.24 | C&P GUI | Good |
   2.24 | Import GUI | Good |
   2.24 | curl /api/metaData | LF's removed; breaks javascript |
   2.24 | curl /api/24/metadata | LF's removed; breaks javascript |
   2.23 | C&P GUI | Good |
   2.23 | Import GUI | Good |
   2.23 | curl /api/metaData | LF's removed; breaks javascript |
   2.23 | curl /api/23/metadata | LF's removed; breaks javascript |

+---------+-----------------------+---------------------------------+

So this affects both the latest versions of 2.23 and 2.24. When the form is rendered in the dataentry app, after all the alterations done by dhis 2, many LFs are added back in, BUT, the items between the <script> tags are left as is. This will sometimes make the forms not appear at all:

<a
link="href://https://drive.google.com/open?id=0B9eJ8PZ_4csMWTVpcGxWRGg1N0U">JS
Error</a>

The easiest way to recreate this is to take the following file, and
upload it to a vanilla 2.23 or 2.24

/metaData
<a href="https://drive.google.com/open?id=0B9eJ8PZ_4csMZVZ2c3IybU8tazg">Download Here</a>

/##/metadata (notice the case difference)
<a href="https://drive.google.com/open?id=0B9eJ8PZ_4csMeVZVNFhZdGZtbTA">Download here</a>
Note that when using the GUI, all the LFs are preserved.

** Affects: dhis2
Importance: Undecided
Status: New

--
You received this bug notification because you are a member of DHIS 2
developers, which is subscribed to DHIS.
https://bugs.launchpad.net/bugs/1617416

Title:
LFs removed on import, breaking custom JS

Status in DHIS:
New

Bug description:
Hello Devs

  It looks like uploads against both /metaData and /xx/metadata are
  removing LFs in their entirety from <dataEntryForms> htmlcode. I
  prepared a table to show:

So this affects both the latest versions of 2.23 and 2.24. When the form is rendered in the dataentry app, after all the alterations done by dhis 2, many LFs are added back in, BUT, the items between the <script> tags are left as is. This will sometimes make the forms not appear at all:

  <a
  link="href://https://drive.google.com/open?id=0B9eJ8PZ_4csMWTVpcGxWRGg1N0U">JS
  Error</a>

The easiest way to recreate this is to take the following file, and
upload it to a vanilla 2.23 or 2.24

/metaData
<a href="https://drive.google.com/open?id=0B9eJ8PZ_4csMZVZ2c3IybU8tazg">Download Here</a>

  /##/metadata (notice the case difference)
  <a href="https://drive.google.com/open?id=0B9eJ8PZ_4csMeVZVNFhZdGZtbTA">Download here</a>
  Note that when using the GUI, all the LFs are preserved.

To manage notifications about this bug go to:
https://bugs.launchpad.net/dhis2/+bug/1617416/+subscriptions

Timothy_Harding1 · 26 August 2016 18:06

AAAAnd it looks like concurrent spaces are removed from Launchpad and a
tags are not parsed.

···

--
You received this bug notification because you are a member of DHIS 2
developers, which is subscribed to DHIS.
https://bugs.launchpad.net/bugs/1617416

Title:
LFs removed on import, breaking custom JS

Status in DHIS:
New

Bug description:
Hello Devs

  It looks like uploads against both /metaData and /xx/metadata are
  removing LFs in their entirety from <dataEntryForms> htmlcode. I
  prepared a table to show:

So this affects both the latest versions of 2.23 and 2.24. When the form is rendered in the dataentry app, after all the alterations done by dhis 2, many LFs are added back in, BUT, the items between the <script> tags are left as is. This will sometimes make the forms not appear at all:

  <a
  link="href://https://drive.google.com/open?id=0B9eJ8PZ_4csMWTVpcGxWRGg1N0U">JS
  Error</a>

The easiest way to recreate this is to take the following file, and
upload it to a vanilla 2.23 or 2.24

/metaData
<a href="https://drive.google.com/open?id=0B9eJ8PZ_4csMZVZ2c3IybU8tazg">Download Here</a>

  /##/metadata (notice the case difference)
  <a href="https://drive.google.com/open?id=0B9eJ8PZ_4csMeVZVNFhZdGZtbTA">Download here</a>
  Note that when using the GUI, all the LFs are preserved.

To manage notifications about this bug go to:
https://bugs.launchpad.net/dhis2/+bug/1617416/+subscriptions

Morten_Olav_Hansen88 · 27 August 2016 08:54

Thanks for reporting.. I haven't seen this issue before, but will look
into it This is all done by the Jackson library.. so we might have to
tweak some settings.

** Changed in: dhis2
Assignee: (unassigned) => Morten Olav Hansen (mortenoh)

···

--
You received this bug notification because you are a member of DHIS 2
developers, which is subscribed to DHIS.
https://bugs.launchpad.net/bugs/1617416

Title:
LFs removed on import, breaking custom JS

Status in DHIS:
New

Bug description:
Hello Devs

  It looks like uploads against both /metaData and /xx/metadata are
  removing LFs in their entirety from <dataEntryForms> htmlcode. I
  prepared a table to show:

So this affects both the latest versions of 2.23 and 2.24. When the form is rendered in the dataentry app, after all the alterations done by dhis 2, many LFs are added back in, BUT, the items between the <script> tags are left as is. This will sometimes make the forms not appear at all:

  <a
  link="href://https://drive.google.com/open?id=0B9eJ8PZ_4csMWTVpcGxWRGg1N0U">JS
  Error</a>

The easiest way to recreate this is to take the following file, and
upload it to a vanilla 2.23 or 2.24

/metaData
<a href="https://drive.google.com/open?id=0B9eJ8PZ_4csMZVZ2c3IybU8tazg">Download Here</a>

  /##/metadata (notice the case difference)
  <a href="https://drive.google.com/open?id=0B9eJ8PZ_4csMeVZVNFhZdGZtbTA">Download here</a>
  Note that when using the GUI, all the LFs are preserved.

To manage notifications about this bug go to:
https://bugs.launchpad.net/dhis2/+bug/1617416/+subscriptions

Morten_Olav_Hansen88 · 27 August 2016 09:05

Hi

Could you retry this with --data-binary instead of -d ?

···

--
You received this bug notification because you are a member of DHIS 2
developers, which is subscribed to DHIS.
https://bugs.launchpad.net/bugs/1617416

Title:
LFs removed on import, breaking custom JS

Status in DHIS:
New

Bug description:
Hello Devs

  It looks like uploads against both /metaData and /xx/metadata are
  removing LFs in their entirety from <dataEntryForms> htmlcode. I
  prepared a table to show:

So this affects both the latest versions of 2.23 and 2.24. When the form is rendered in the dataentry app, after all the alterations done by dhis 2, many LFs are added back in, BUT, the items between the <script> tags are left as is. This will sometimes make the forms not appear at all:

  <a
  link="href://https://drive.google.com/open?id=0B9eJ8PZ_4csMWTVpcGxWRGg1N0U">JS
  Error</a>

The easiest way to recreate this is to take the following file, and
upload it to a vanilla 2.23 or 2.24

/metaData
<a href="https://drive.google.com/open?id=0B9eJ8PZ_4csMZVZ2c3IybU8tazg">Download Here</a>

  /##/metadata (notice the case difference)
  <a href="https://drive.google.com/open?id=0B9eJ8PZ_4csMeVZVNFhZdGZtbTA">Download here</a>
  Note that when using the GUI, all the LFs are preserved.

To manage notifications about this bug go to:
https://bugs.launchpad.net/dhis2/+bug/1617416/+subscriptions

Morten_Olav_Hansen88 · 27 August 2016 09:05

I mean for cURL

···

--
You received this bug notification because you are a member of DHIS 2
developers, which is subscribed to DHIS.
https://bugs.launchpad.net/bugs/1617416

Title:
LFs removed on import, breaking custom JS

Status in DHIS:
New

Bug description:
Hello Devs

  It looks like uploads against both /metaData and /xx/metadata are
  removing LFs in their entirety from <dataEntryForms> htmlcode. I
  prepared a table to show:

So this affects both the latest versions of 2.23 and 2.24. When the form is rendered in the dataentry app, after all the alterations done by dhis 2, many LFs are added back in, BUT, the items between the <script> tags are left as is. This will sometimes make the forms not appear at all:

  <a
  link="href://https://drive.google.com/open?id=0B9eJ8PZ_4csMWTVpcGxWRGg1N0U">JS
  Error</a>

The easiest way to recreate this is to take the following file, and
upload it to a vanilla 2.23 or 2.24

/metaData
<a href="https://drive.google.com/open?id=0B9eJ8PZ_4csMZVZ2c3IybU8tazg">Download Here</a>

  /##/metadata (notice the case difference)
  <a href="https://drive.google.com/open?id=0B9eJ8PZ_4csMeVZVNFhZdGZtbTA">Download here</a>
  Note that when using the GUI, all the LFs are preserved.

To manage notifications about this bug go to:
https://bugs.launchpad.net/dhis2/+bug/1617416/+subscriptions

Timothy_Harding1 · 29 August 2016 13:21

Ah, looks like that was the problem.

Going to test it some more today, thanks for pointing that out. Now I need
to go find the curl dev who thought it was a good idea to munge EOL chars
from files *by default*.
[image: Inline image 1]

Thank you very much Morten,
The stack traces I sent you directly were an unrelated problem with my curl
command, and I got further stack traces when I sent over a .zip as curl
decided to remove the EOL characters from the binary zip data...

*Timothy Harding*
Sr. Systems Analyst, BAO Systems
+1 202-536-1541 | tharding@baosystems.com | http://www.baosystems.com | Skype:
hardingt@gmail.com | 2900 K Street, Suite 404, Washington D.C. 20007

···

On Sat, Aug 27, 2016 at 5:05 AM, Morten Olav Hansen < 1617416@bugs.launchpad.net> wrote:

I mean for cURL

--
You received this bug notification because you are subscribed to the bug
report.
https://bugs.launchpad.net/bugs/1617416

Title:
  LFs removed on import, breaking custom JS

Status in DHIS:
  New

Bug description:
  Hello Devs

  It looks like uploads against both /metaData and /xx/metadata are
  removing LFs in their entirety from <dataEntryForms> htmlcode. I
  prepared a table to show:

  +---------+-----------------------+---------------------------------+
  > Version | Upload Method | Result |
  +---------+-----------------------+---------------------------------+
  > 2.24 | C&P GUI | Good |
  > 2.24 | Import GUI | Good |
  > 2.24 | curl /api/metaData | LF's removed; breaks javascript |
  > 2.24 | curl /api/24/metadata | LF's removed; breaks javascript |
  > 2.23 | C&P GUI | Good |
  > 2.23 | Import GUI | Good |
  > 2.23 | curl /api/metaData | LF's removed; breaks javascript |
  > 2.23 | curl /api/23/metadata | LF's removed; breaks javascript |
  +---------+-----------------------+---------------------------------+

  So this affects both the latest versions of 2.23 and 2.24. When the form
is rendered in the dataentry app, after all the alterations done by dhis 2,
many LFs are added back in, BUT, the items between the <script> tags are
left as is. This will sometimes make the forms not appear at all:

  <a
  link="href://https://drive.google.com/open?id=0B9eJ8PZ_
4csMWTVpcGxWRGg1N0U">JS
  Error</a>

  The easiest way to recreate this is to take the following file, and
  upload it to a vanilla 2.23 or 2.24

  /metaData
  <a href="https://drive.google.com/open?id=0B9eJ8PZ_4csMZVZ2c3IybU8tazg">Download
Here</a>

  /##/metadata (notice the case difference)
  <a href="https://drive.google.com/open?id=0B9eJ8PZ_4csMeVZVNFhZdGZtbTA">Download
here</a>
  Note that when using the GUI, all the LFs are preserved.

To manage notifications about this bug go to:
https://bugs.launchpad.net/dhis2/+bug/1617416/+subscriptions

** Attachment added: "facepalm.png"
https://bugs.launchpad.net/bugs/1617416/+attachment/4730032/+files/facepalm.png

** Attachment added: "PastedGraphic-1.png"
https://bugs.launchpad.net/bugs/1617416/+attachment/4730033/+files/PastedGraphic-1.png

--
You received this bug notification because you are a member of DHIS 2
developers, which is subscribed to DHIS.
https://bugs.launchpad.net/bugs/1617416

Title:
LFs removed on import, breaking custom JS

Status in DHIS:
New

Bug description:
Hello Devs

  It looks like uploads against both /metaData and /xx/metadata are
  removing LFs in their entirety from <dataEntryForms> htmlcode. I
  prepared a table to show:

So this affects both the latest versions of 2.23 and 2.24. When the form is rendered in the dataentry app, after all the alterations done by dhis 2, many LFs are added back in, BUT, the items between the <script> tags are left as is. This will sometimes make the forms not appear at all:

  <a
  link="href://https://drive.google.com/open?id=0B9eJ8PZ_4csMWTVpcGxWRGg1N0U">JS
  Error</a>

The easiest way to recreate this is to take the following file, and
upload it to a vanilla 2.23 or 2.24

/metaData
<a href="https://drive.google.com/open?id=0B9eJ8PZ_4csMZVZ2c3IybU8tazg">Download Here</a>

  /##/metadata (notice the case difference)
  <a href="https://drive.google.com/open?id=0B9eJ8PZ_4csMeVZVNFhZdGZtbTA">Download here</a>
  Note that when using the GUI, all the LFs are preserved.

To manage notifications about this bug go to:
https://bugs.launchpad.net/dhis2/+bug/1617416/+subscriptions