Does anyone have some idea on how to troubleshoot the table analytics failure process as indicated by the image below? The process starts fine but it is stuck after a while. Then, the data called from the crosstab visualization could not be updated according to data entered in the database. Before, it has been already working fine and scheduled for every day.
Using DHIS2 2.19.
Thank you !
···
Tantely Raminosoa
Data Officer
Management Sciences for Health
Antananarivo Madagascar
I think you need to have a look at your server logs, there will be full stack trace etc which we can use in debugging your issues
···
On Fri, Jun 17, 2016 at 1:45 PM, Raminosoa Rabemanantsoa, Tantely traminosoa@mikolo.org wrote:
Hello,
Does anyone have some idea on how to troubleshoot the table analytics failure process as indicated by the image below? The process starts fine but it is stuck after a while. Then, the data called from the crosstab visualization could not be updated according to data entered in the database. Before, it has been already working fine and scheduled for every day.
Using DHIS2 2.19.
Thank you !
– Tantely Raminosoa
Data Officer
Management Sciences for Health
Antananarivo Madagascar
This message and its attachments are confidential and solely for the intended recipients. If received in error, please delete them and notify the sender via reply e-mail immediately.
Please find attached an extraction of the log file containing the process of the table analytics update. This kind of error is always repeating “Caused by: org.postgresql.util.PSQLException: ERROR: could not extend file “base/16387/820574”: No space left on device”. The memory allocated to Tomcat and postgreSQL are already tuned according to HISP recommendations. What are the other parameters that should be corrected in order to troubleshoot those issues?
Tantely Raminosoa Data Officer
Management Sciences for Health
Antananarivo Madagascar
Mobile: 0321180040
E-mail: traminosoa@mikolo.org
Skype: traminosoa
Stronger health systems. Greater health impact. www.msh.org
On Fri, Jun 17, 2016 at 10:32 AM, Morten Olav Hansen morten@dhis2.org wrote:
Hi
I think you need to have a look at your server logs, there will be full stack trace etc which we can use in debugging your issues
On Fri, Jun 17, 2016 at 1:45 PM, Raminosoa Rabemanantsoa, Tantely traminosoa@mikolo.org wrote:
Hello,
Does anyone have some idea on how to troubleshoot the table analytics failure process as indicated by the image below? The process starts fine but it is stuck after a while. Then, the data called from the crosstab visualization could not be updated according to data entered in the database. Before, it has been already working fine and scheduled for every day.
Using DHIS2 2.19.
Thank you !
– Tantely Raminosoa
Data Officer
Management Sciences for Health
Antananarivo Madagascar
This message and its attachments are confidential and solely for the intended recipients. If received in error, please delete them and notify the sender via reply e-mail immediately.
Oh, this have nothing to do with DHIS 2, it means you harddrive is out of space… maybe you should have a look at your tomcat/logs directory, see if its full (we have had some issues with logs filling up drives), if not, you need to somehow free some disk space on your drive.
···
On Fri, Jun 17, 2016 at 4:03 PM, Raminosoa Rabemanantsoa, Tantely traminosoa@mikolo.org wrote:
Hi Morten,
Please find attached an extraction of the log file containing the process of the table analytics update. This kind of error is always repeating “Caused by: org.postgresql.util.PSQLException: ERROR: could not extend file “base/16387/820574”: No space left on device”. The memory allocated to Tomcat and postgreSQL are already tuned according to HISP recommendations. What are the other parameters that should be corrected in order to troubleshoot those issues?
Thank you for your support.
– Tantely Raminosoa Data Officer
Management Sciences for Health
Antananarivo Madagascar
Mobile: 0321180040
E-mail: traminosoa@mikolo.org
Skype: traminosoa
Stronger health systems. Greater health impact. www.msh.org
This message and its attachments are confidential and solely for the intended recipients. If received in error, please delete them and notify the sender via reply e-mail immediately.
On Fri, Jun 17, 2016 at 1:45 PM, Raminosoa Rabemanantsoa, Tantely traminosoa@mikolo.org wrote:
Hello,
Does anyone have some idea on how to troubleshoot the table analytics failure process as indicated by the image below? The process starts fine but it is stuck after a while. Then, the data called from the crosstab visualization could not be updated according to data entered in the database. Before, it has been already working fine and scheduled for every day.
Using DHIS2 2.19.
Thank you !
– Tantely Raminosoa
Data Officer
Management Sciences for Health
Antananarivo Madagascar
This message and its attachments are confidential and solely for the intended recipients. If received in error, please delete them and notify the sender via reply e-mail immediately.
it seems you are running out of disk space on your computer.
regards,
Lars
em.util.ConcurrentUtils.waitForCompletion(ConcurrentUtils.java:51) ... 13 more Caused by: org.springframework.dao.DataAccessResourceFailureException: StatementCallback; SQL [create index "in_sixmonthlyapril_ax_2016_RpwHm" on analytics_temp_2016 ("sixmonthlyapril")]; ERROR: could not write block 3842 of temporary file: No space left on device Hint: Perhaps out of disk space?; nested exception is org.postgresql.util.PSQLException: ERROR: could not write block 3842 of temporary file: No space left on device Hint: Perhaps out of disk space?
···
On Fri, Jun 17, 2016 at 11:03 AM, Raminosoa Rabemanantsoa, Tantely traminosoa@mikolo.org wrote:
Hi Morten,
Please find attached an extraction of the log file containing the process of the table analytics update. This kind of error is always repeating “Caused by: org.postgresql.util.PSQLException: ERROR: could not extend file “base/16387/820574”: No space left on device”. The memory allocated to Tomcat and postgreSQL are already tuned according to HISP recommendations. What are the other parameters that should be corrected in order to troubleshoot those issues?
Thank you for your support.
– Tantely Raminosoa Data Officer
Management Sciences for Health
Antananarivo Madagascar
Mobile: 0321180040
E-mail: traminosoa@mikolo.org
Skype: traminosoa
Stronger health systems. Greater health impact. www.msh.org
This message and its attachments are confidential and solely for the intended recipients. If received in error, please delete them and notify the sender via reply e-mail immediately.
On Fri, Jun 17, 2016 at 1:45 PM, Raminosoa Rabemanantsoa, Tantely traminosoa@mikolo.org wrote:
Hello,
Does anyone have some idea on how to troubleshoot the table analytics failure process as indicated by the image below? The process starts fine but it is stuck after a while. Then, the data called from the crosstab visualization could not be updated according to data entered in the database. Before, it has been already working fine and scheduled for every day.
Using DHIS2 2.19.
Thank you !
– Tantely Raminosoa
Data Officer
Management Sciences for Health
Antananarivo Madagascar
This message and its attachments are confidential and solely for the intended recipients. If received in error, please delete them and notify the sender via reply e-mail immediately.
check your disk space by this command if u are on linux: df -h
That happens to me a lot of time when i do work on smaller Virtual Machines.
regards,
···
On Fri, Jun 17, 2016 at 11:13 AM, Lars Helge Øverland lars@dhis2.org wrote:
Hi Raminosoa,
it seems you are running out of disk space on your computer.
regards,
Lars
em.util.ConcurrentUtils.waitForCompletion(ConcurrentUtils.java:51) ... 13 more Caused by: org.springframework.dao.DataAccessResourceFailureException: StatementCallback; SQL [create index "in_sixmonthlyapril_ax_2016_RpwHm" on analytics_temp_2016 ("sixmonthlyapril")]; ERROR: could not write block 3842 of temporary file: No space left on device Hint: Perhaps out of disk space?; nested exception is org.postgresql.util.PSQLException: ERROR: could not write block 3842 of temporary file: No space left on device Hint: Perhaps out of disk space?
On Fri, Jun 17, 2016 at 11:03 AM, Raminosoa Rabemanantsoa, Tantely traminosoa@mikolo.org wrote:
Hi Morten,
Please find attached an extraction of the log file containing the process of the table analytics update. This kind of error is always repeating “Caused by: org.postgresql.util.PSQLException: ERROR: could not extend file “base/16387/820574”: No space left on device”. The memory allocated to Tomcat and postgreSQL are already tuned according to HISP recommendations. What are the other parameters that should be corrected in order to troubleshoot those issues?
Thank you for your support.
– Tantely Raminosoa Data Officer
Management Sciences for Health
Antananarivo Madagascar
Mobile: 0321180040
E-mail: traminosoa@mikolo.org
Skype: traminosoa
Stronger health systems. Greater health impact. www.msh.org
This message and its attachments are confidential and solely for the intended recipients. If received in error, please delete them and notify the sender via reply e-mail immediately.
On Fri, Jun 17, 2016 at 1:45 PM, Raminosoa Rabemanantsoa, Tantely traminosoa@mikolo.org wrote:
Hello,
Does anyone have some idea on how to troubleshoot the table analytics failure process as indicated by the image below? The process starts fine but it is stuck after a while. Then, the data called from the crosstab visualization could not be updated according to data entered in the database. Before, it has been already working fine and scheduled for every day.
Using DHIS2 2.19.
Thank you !
– Tantely Raminosoa
Data Officer
Management Sciences for Health
Antananarivo Madagascar
This message and its attachments are confidential and solely for the intended recipients. If received in error, please delete them and notify the sender via reply e-mail immediately.
Thank you for your points of view. I am just wondering on this disk capacity issue. We are using a cloud Amazon Web services to host the DHIS2 application. There are 80 gigabyte disk space used only by DHIS2. There are 30 gigabyte of RAM where 9 GB are for JVM and 9 GB for postgreSQL. My question is that which parameters should I correct to troubleshoot this disk issue as physically there is still large available disk space on the server, or is there other parameters to allocate the physical disk space (in addition to the RAM) managed by JVM?
Thank you.
Tantely.
This message and its attachments are confidential and solely for the intended recipients. If received in error, please delete them and notify the sender via reply e-mail immediately.
This is an issue with Postgres specifically and is therefore not related to the JVM. Most likely it is failing to temporarily store data during a big query due to low capacity.
Specifically see section 18.4.2 which addresses the temp_file_limit.
Also try to do a VACUUM FULL
···
On Fri, Jun 17, 2016 at 3:26 PM, Raminosoa Rabemanantsoa, Tantely traminosoa@mikolo.org wrote:
Hi All,
Thank you for your points of view. I am just wondering on this disk capacity issue. We are using a cloud Amazon Web services to host the DHIS2 application. There are 80 gigabyte disk space used only by DHIS2. There are 30 gigabyte of RAM where 9 GB are for JVM and 9 GB for postgreSQL. My question is that which parameters should I correct to troubleshoot this disk issue as physically there is still large available disk space on the server, or is there other parameters to allocate the physical disk space (in addition to the RAM) managed by JVM?
Thank you.
Tantely.
This message and its attachments are confidential and solely for the intended recipients. If received in error, please delete them and notify the sender via reply e-mail immediately.
We have experienced similar issues earlier where we were having plenty of disk space but most of it was not mounted. Could you please check the mount points and then check the disk space available for postgres and other stuff. You can use the command ‘df’ and understand how is the disk organized.
Regards,
Sultan Ahamar.
···
On Fri, Jun 17, 2016 at 7:13 PM, Halvdan Hoem Grelland halvdan@dhis2.org wrote:
This is an issue with Postgres specifically and is therefore not related to the JVM. Most likely it is failing to temporarily store data during a big query due to low capacity.
On Fri, Jun 17, 2016 at 3:26 PM, Raminosoa Rabemanantsoa, Tantely traminosoa@mikolo.org wrote:
Hi All,
Thank you for your points of view. I am just wondering on this disk capacity issue. We are using a cloud Amazon Web services to host the DHIS2 application. There are 80 gigabyte disk space used only by DHIS2. There are 30 gigabyte of RAM where 9 GB are for JVM and 9 GB for postgreSQL. My question is that which parameters should I correct to troubleshoot this disk issue as physically there is still large available disk space on the server, or is there other parameters to allocate the physical disk space (in addition to the RAM) managed by JVM?
Thank you.
Tantely.
This message and its attachments are confidential and solely for the intended recipients. If received in error, please delete them and notify the sender via reply e-mail immediately.
Thank you for your suggestions. I am on the way to verify the server and I will be back with informations.
Thank you.
This message and its attachments are confidential and solely for the intended recipients. If received in error, please delete them and notify the sender via reply e-mail immediately.
We have experienced similar issues earlier where we were having plenty of disk space but most of it was not mounted. Could you please check the mount points and then check the disk space available for postgres and other stuff. You can use the command ‘df’ and understand how is the disk organized.
Regards,
Sultan Ahamar.
–
On Fri, Jun 17, 2016 at 7:13 PM, Halvdan Hoem Grelland halvdan@dhis2.org wrote:
This is an issue with Postgres specifically and is therefore not related to the JVM. Most likely it is failing to temporarily store data during a big query due to low capacity.
On Fri, Jun 17, 2016 at 3:26 PM, Raminosoa Rabemanantsoa, Tantely traminosoa@mikolo.org wrote:
Hi All,
Thank you for your points of view. I am just wondering on this disk capacity issue. We are using a cloud Amazon Web services to host the DHIS2 application. There are 80 gigabyte disk space used only by DHIS2. There are 30 gigabyte of RAM where 9 GB are for JVM and 9 GB for postgreSQL. My question is that which parameters should I correct to troubleshoot this disk issue as physically there is still large available disk space on the server, or is there other parameters to allocate the physical disk space (in addition to the RAM) managed by JVM?
Thank you.
Tantely.
This message and its attachments are confidential and solely for the intended recipients. If received in error, please delete them and notify the sender via reply e-mail immediately.
