[Bug 1567901] [NEW] Analytics Indexing Failure - long text

Support - Assistance technique
1

Public bug reported:

DHIS 2.21 when performing an analytics update, analytics fails, returning the error:

···

-----------------------------------------------------------------------
* ERROR 2016-04-08 13:29:32,627 Unexpected error occurred in scheduled task. (TaskUtils.java [taskScheduler-2])
java.lang.RuntimeException: Exception during execution
...
Caused by: java.util.concurrent.ExecutionException: org.springframework.dao.DataAccessResourceFailureException: StatementCallback; SQL [create index "in_ciWR6s18VWH_ax_event_1980_dvqwq9jgpn8_M0WjJ" on analytics_event_temp_1980_dvqwq9jgpn8 ("ciWR6s18VWH")]; ERROR: index row size 3512 exceeds maximum 2712 for index "in_ciWR6s18VWH_ax_event_1980_dvqwq9jgpn8_M0WjJ"
  Hint: Values larger than 1/3 of a buffer page cannot be indexed.
-----------------------------------------------------------------------

Notes:
This has not been checked on 2.22.
This has been replicated on any browser version. Server is Ubuntu 14.04 LTS, Tomcat 8, Java 8.
Recommend turning off indexing for long-text values, or using a set-length of some part of the string for the purposes of indexing.

** Affects: dhis2
     Importance: Undecided
         Status: New

--
You received this bug notification because you are a member of DHIS 2
developers, which is subscribed to DHIS.
https://bugs.launchpad.net/bugs/1567901

Title:
  Analytics Indexing Failure - long text

Status in DHIS:
  New

Bug description:
  DHIS 2.21 when performing an analytics update, analytics fails, returning the error:
  -----------------------------------------------------------------------
  * ERROR 2016-04-08 13:29:32,627 Unexpected error occurred in scheduled task. (TaskUtils.java [taskScheduler-2])
  java.lang.RuntimeException: Exception during execution
  ...
  Caused by: java.util.concurrent.ExecutionException: org.springframework.dao.DataAccessResourceFailureException: StatementCallback; SQL [create index "in_ciWR6s18VWH_ax_event_1980_dvqwq9jgpn8_M0WjJ" on analytics_event_temp_1980_dvqwq9jgpn8 ("ciWR6s18VWH")]; ERROR: index row size 3512 exceeds maximum 2712 for index "in_ciWR6s18VWH_ax_event_1980_dvqwq9jgpn8_M0WjJ"
    Hint: Values larger than 1/3 of a buffer page cannot be indexed.
  -----------------------------------------------------------------------

  Notes:
  This has not been checked on 2.22.
  This has been replicated on any browser version. Server is Ubuntu 14.04 LTS, Tomcat 8, Java 8.
  Recommend turning off indexing for long-text values, or using a set-length of some part of the string for the purposes of indexing.

To manage notifications about this bug go to:
https://bugs.launchpad.net/dhis2/+bug/1567901/+subscriptions

2

** Changed in: dhis2
    Milestone: None => 2.23

** Changed in: dhis2
     Assignee: (unassigned) => Lars Helge Øverland (larshelge)

** Changed in: dhis2
   Importance: Undecided => Medium

···

--
You received this bug notification because you are a member of DHIS 2
developers, which is subscribed to DHIS.
https://bugs.launchpad.net/bugs/1567901

Title:
  Analytics Indexing Failure - long text

Status in DHIS:
  New

Bug description:
  DHIS 2.21 when performing an analytics update, analytics fails, returning the error:
  -----------------------------------------------------------------------
  * ERROR 2016-04-08 13:29:32,627 Unexpected error occurred in scheduled task. (TaskUtils.java [taskScheduler-2])
  java.lang.RuntimeException: Exception during execution
  ...
  Caused by: java.util.concurrent.ExecutionException: org.springframework.dao.DataAccessResourceFailureException: StatementCallback; SQL [create index "in_ciWR6s18VWH_ax_event_1980_dvqwq9jgpn8_M0WjJ" on analytics_event_temp_1980_dvqwq9jgpn8 ("ciWR6s18VWH")]; ERROR: index row size 3512 exceeds maximum 2712 for index "in_ciWR6s18VWH_ax_event_1980_dvqwq9jgpn8_M0WjJ"
    Hint: Values larger than 1/3 of a buffer page cannot be indexed.
  -----------------------------------------------------------------------

  Notes:
  This has not been checked on 2.22.
  This has been replicated on any browser version. Server is Ubuntu 14.04 LTS, Tomcat 8, Java 8.
  Recommend turning off indexing for long-text values, or using a set-length of some part of the string for the purposes of indexing.

To manage notifications about this bug go to:
https://bugs.launchpad.net/dhis2/+bug/1567901/+subscriptions

3

Think I agree here. We could skip indexing of non-aggregateable data
elements/attributes per now entirely (including text value type) as we
cannot do analytics on it.

We could look into text indexing (lucene) but that is for later.

···

--
You received this bug notification because you are a member of DHIS 2
developers, which is subscribed to DHIS.
https://bugs.launchpad.net/bugs/1567901

Title:
  Analytics Indexing Failure - long text

Status in DHIS:
  New

Bug description:
  DHIS 2.21 when performing an analytics update, analytics fails, returning the error:
  -----------------------------------------------------------------------
  * ERROR 2016-04-08 13:29:32,627 Unexpected error occurred in scheduled task. (TaskUtils.java [taskScheduler-2])
  java.lang.RuntimeException: Exception during execution
  ...
  Caused by: java.util.concurrent.ExecutionException: org.springframework.dao.DataAccessResourceFailureException: StatementCallback; SQL [create index "in_ciWR6s18VWH_ax_event_1980_dvqwq9jgpn8_M0WjJ" on analytics_event_temp_1980_dvqwq9jgpn8 ("ciWR6s18VWH")]; ERROR: index row size 3512 exceeds maximum 2712 for index "in_ciWR6s18VWH_ax_event_1980_dvqwq9jgpn8_M0WjJ"
    Hint: Values larger than 1/3 of a buffer page cannot be indexed.
  -----------------------------------------------------------------------

  Notes:
  This has not been checked on 2.22.
  This has been replicated on any browser version. Server is Ubuntu 14.04 LTS, Tomcat 8, Java 8.
  Recommend turning off indexing for long-text values, or using a set-length of some part of the string for the purposes of indexing.

To manage notifications about this bug go to:
https://bugs.launchpad.net/dhis2/+bug/1567901/+subscriptions

4

Impl a fix in trunk (2.23-SNAPSHOT) where data elements and attributes
with value type LONG_TEXT and without option sets are not indexed in the
event analytics tables.

** Changed in: dhis2
       Status: New => Fix Committed

···

--
You received this bug notification because you are a member of DHIS 2
developers, which is subscribed to DHIS.
https://bugs.launchpad.net/bugs/1567901

Title:
  Analytics Indexing Failure - long text

Status in DHIS:
  Fix Committed

Bug description:
  DHIS 2.21 when performing an analytics update, analytics fails, returning the error:
  -----------------------------------------------------------------------
  * ERROR 2016-04-08 13:29:32,627 Unexpected error occurred in scheduled task. (TaskUtils.java [taskScheduler-2])
  java.lang.RuntimeException: Exception during execution
  ...
  Caused by: java.util.concurrent.ExecutionException: org.springframework.dao.DataAccessResourceFailureException: StatementCallback; SQL [create index "in_ciWR6s18VWH_ax_event_1980_dvqwq9jgpn8_M0WjJ" on analytics_event_temp_1980_dvqwq9jgpn8 ("ciWR6s18VWH")]; ERROR: index row size 3512 exceeds maximum 2712 for index "in_ciWR6s18VWH_ax_event_1980_dvqwq9jgpn8_M0WjJ"
    Hint: Values larger than 1/3 of a buffer page cannot be indexed.
  -----------------------------------------------------------------------

  Notes:
  This has not been checked on 2.22.
  This has been replicated on any browser version. Server is Ubuntu 14.04 LTS, Tomcat 8, Java 8.
  Recommend turning off indexing for long-text values, or using a set-length of some part of the string for the purposes of indexing.

To manage notifications about this bug go to:
https://bugs.launchpad.net/dhis2/+bug/1567901/+subscriptions

5

Lars,

Thanks for fixing this - any chance of the fix being backported to 2.22 and/or 2.21?

Regards

Calle

···

On 12 April 2016 at 18:54, Lars Helge Øverland lars@dhis2.org wrote:

Impl a fix in trunk (2.23-SNAPSHOT) where data elements and attributes

with value type LONG_TEXT and without option sets are not indexed in the

event analytics tables.

** Changed in: dhis2

   Status: New => Fix Committed

–

You received this bug notification because you are a member of DHIS 2

developers, which is subscribed to DHIS.

https://bugs.launchpad.net/bugs/1567901

Title:

Analytics Indexing Failure - long text

Status in DHIS:

Fix Committed

Bug description:

DHIS 2.21 when performing an analytics update, analytics fails, returning the error:

  • ERROR 2016-04-08 13:29:32,627 Unexpected error occurred in scheduled task. (TaskUtils.java [taskScheduler-2])

java.lang.RuntimeException: Exception during execution

…

Caused by: java.util.concurrent.ExecutionException: org.springframework.dao.DataAccessResourceFailureException: StatementCallback; SQL [create index “in_ciWR6s18VWH_ax_event_1980_dvqwq9jgpn8_M0WjJ” on analytics_event_temp_1980_dvqwq9jgpn8 (“ciWR6s18VWH”)]; ERROR: index row size 3512 exceeds maximum 2712 for index “in_ciWR6s18VWH_ax_event_1980_dvqwq9jgpn8_M0WjJ”

Hint: Values larger than 1/3 of a buffer page cannot be indexed.

Notes:

This has not been checked on 2.22.

This has been replicated on any browser version. Server is Ubuntu 14.04 LTS, Tomcat 8, Java 8.

Recommend turning off indexing for long-text values, or using a set-length of some part of the string for the purposes of indexing.

To manage notifications about this bug go to:

https://bugs.launchpad.net/dhis2/+bug/1567901/+subscriptions

Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp

–

Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg

6

Hi Calle,

in this case we have refactored much the analytics table generation code,
so back-porting means a completely new fix, so I think we will not this
right now. 2.23 will be out very shortly, hopefully you can upgrade to 2.23
to resolve this.

regards,

Lars

···

On Wed, Apr 13, 2016 at 11:25 AM, Calle Hedberg <calle.hedberg@gmail.com> wrote:

Lars,

Thanks for fixing this - any chance of the fix being backported to 2.22
and/or 2.21?

Regards
Calle

On 12 April 2016 at 18:54, Lars Helge Øverland <lars@dhis2.org> wrote:

Impl a fix in trunk (2.23-SNAPSHOT) where data elements and attributes
with value type LONG_TEXT and without option sets are not indexed in the
event analytics tables.

** Changed in: dhis2
       Status: New => Fix Committed

--
You received this bug notification because you are a member of DHIS 2
developers, which is subscribed to DHIS.
https://bugs.launchpad.net/bugs/1567901

Title:
  Analytics Indexing Failure - long text

Status in DHIS:
  Fix Committed

Bug description:
  DHIS 2.21 when performing an analytics update, analytics fails,
returning the error:
  -----------------------------------------------------------------------
  * ERROR 2016-04-08 13:29:32,627 Unexpected error occurred in scheduled
task. (TaskUtils.java [taskScheduler-2])
  java.lang.RuntimeException: Exception during execution
  ...
  Caused by: java.util.concurrent.ExecutionException:
org.springframework.dao.DataAccessResourceFailureException:
StatementCallback; SQL [create index
"in_ciWR6s18VWH_ax_event_1980_dvqwq9jgpn8_M0WjJ" on
analytics_event_temp_1980_dvqwq9jgpn8 ("ciWR6s18VWH")]; ERROR: index row
size 3512 exceeds maximum 2712 for index
"in_ciWR6s18VWH_ax_event_1980_dvqwq9jgpn8_M0WjJ"
    Hint: Values larger than 1/3 of a buffer page cannot be indexed.
  -----------------------------------------------------------------------

  Notes:
  This has not been checked on 2.22.
  This has been replicated on any browser version. Server is Ubuntu
14.04 LTS, Tomcat 8, Java 8.
  Recommend turning off indexing for long-text values, or using a
set-length of some part of the string for the purposes of indexing.

To manage notifications about this bug go to:
https://bugs.launchpad.net/dhis2/+bug/1567901/+subscriptions

_______________________________________________
Mailing list: DHIS 2 developers in Launchpad
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : DHIS 2 developers in Launchpad
More help : ListHelp - Launchpad Help

--

*******************************************

Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@gmail.com

Skype: calle_hedberg

*******************************************

_______________________________________________
Mailing list: DHIS 2 developers in Launchpad
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : DHIS 2 developers in Launchpad
More help : ListHelp - Launchpad Help

--
Lars Helge Øverland
Lead developer, DHIS 2
University of Oslo
Skype: larshelgeoverland
http://www.dhis2.org <https://www.dhis2.org/&gt;

--
You received this bug notification because you are a member of DHIS 2
developers, which is subscribed to DHIS.
https://bugs.launchpad.net/bugs/1567901

Title:
  Analytics Indexing Failure - long text

Status in DHIS:
  Fix Committed

Bug description:
  DHIS 2.21 when performing an analytics update, analytics fails, returning the error:
  -----------------------------------------------------------------------
  * ERROR 2016-04-08 13:29:32,627 Unexpected error occurred in scheduled task. (TaskUtils.java [taskScheduler-2])
  java.lang.RuntimeException: Exception during execution
  ...
  Caused by: java.util.concurrent.ExecutionException: org.springframework.dao.DataAccessResourceFailureException: StatementCallback; SQL [create index "in_ciWR6s18VWH_ax_event_1980_dvqwq9jgpn8_M0WjJ" on analytics_event_temp_1980_dvqwq9jgpn8 ("ciWR6s18VWH")]; ERROR: index row size 3512 exceeds maximum 2712 for index "in_ciWR6s18VWH_ax_event_1980_dvqwq9jgpn8_M0WjJ"
    Hint: Values larger than 1/3 of a buffer page cannot be indexed.
  -----------------------------------------------------------------------

  Notes:
  This has not been checked on 2.22.
  This has been replicated on any browser version. Server is Ubuntu 14.04 LTS, Tomcat 8, Java 8.
  Recommend turning off indexing for long-text values, or using a set-length of some part of the string for the purposes of indexing.

To manage notifications about this bug go to:
https://bugs.launchpad.net/dhis2/+bug/1567901/+subscriptions