I have some major problems when I am trying to import meta-data and data from one DHIS2 instance to another. These are as follows:
Issue 1:
I have a DHIS2 installation running on a VMWare Virtual Machine. Using the Meta-Data Export/Import features in DHIS2, I have imported data from another DHIS2 instance running on a laptop into the VM “successfully”. “Successfully” in quotes because the log says it was imported successfully. However, when I shut down the VM and start it again, all the imported data is missing from the VM. That is, it is as if I did not import any data. In the data import screen, Dry Run option is set to “No”. I did take a snapshot of the VM immediately after importation and reverted to the snapshot after firing up the VM but nothing doing. Any ideas on why the data is not persisting on the VM?
Note: Nothing I am doing on the VM persists! For example, I looked at the default user I have been using to access DHIS2 (admin:district) and added all the roles to the user. These additions to the user account were not reflected after restarting the VM.
Important Note: I have noticed, by looking at the logs, that every time the DHIS2 instance starts up, it is attempting to run alot of table altering SQL commands, most of which are failing. I suspect that it is these commands that are interfering with my data. If you would like to see a copy of the log file, I will be happy to provide it.
Issue 2:
After importing the data, I log out of DHIS2 and try logging in using a different user whose info has been imported into the VM. This does not work. This is BEFORE I even shut down the machine. When I log back in using the default admin user, I can see the user accounts in the system.
Note: I have seen from an old posting on the forum that importation of users was disabled. I don’t know if this is still the case and therefore why I am having this problem?
Issue 3:
After importing meta-data, I immediately tried to import DATA from the other instance. Data import fails with a message that it was unable to clear the BatchHandler.
From what you are saying, it sounds like all these issues are caused by using an in-memory db… Are you sure that you are connecting to a database, and that the user has the proper privileges?
Try to look for dhis2 home references in the log… it should be early in the startup
···
–
Morten
On Wed, Dec 18, 2013 at 10:09 AM, Edwin Mulwa eddiemu@gmail.com wrote:
Hi,
I have some major problems when I am trying to import meta-data and data from one DHIS2 instance to another. These are as follows:
Issue 1:
I have a DHIS2 installation running on a VMWare Virtual Machine. Using the Meta-Data Export/Import features in DHIS2, I have imported data from another DHIS2 instance running on a laptop into the VM “successfully”. “Successfully” in quotes because the log says it was imported successfully. However, when I shut down the VM and start it again, all the imported data is missing from the VM. That is, it is as if I did not import any data. In the data import screen, Dry Run option is set to “No”. I did take a snapshot of the VM immediately after importation and reverted to the snapshot after firing up the VM but nothing doing. Any ideas on why the data is not persisting on the VM?
Note: Nothing I am doing on the VM persists! For example, I looked at the default user I have been using to access DHIS2 (admin:district) and added all the roles to the user. These additions to the user account were not reflected after restarting the VM.
Important Note: I have noticed, by looking at the logs, that every time the DHIS2 instance starts up, it is attempting to run alot of table altering SQL commands, most of which are failing. I suspect that it is these commands that are interfering with my data. If you would like to see a copy of the log file, I will be happy to provide it.
Issue 2:
After importing the data, I log out of DHIS2 and try logging in using a different user whose info has been imported into the VM. This does not work. This is BEFORE I even shut down the machine. When I log back in using the default admin user, I can see the user accounts in the system.
Note: I have seen from an old posting on the forum that importation of users was disabled. I don’t know if this is still the case and therefore why I am having this problem?
Issue 3:
After importing meta-data, I immediately tried to import DATA from the other instance. Data import fails with a message that it was unable to clear the BatchHandler.
Thank you! The in-memory db was the cause of one of my problems. So Issue 1 is resolved!
I still cannot log in using an imported user. I suspect is it for the reason I mentioned in my original post i.e. importation of users was disabled?
The other remaining problem is the importation of XML data. It is still giving a “Failed to flush BatchHandler” error when I try to import the data. I have attached the relevant section of the log file so that you check and see if you can make anything of it.
On Wed, Dec 18, 2013 at 1:23 AM, Morten Olav Hansen mortenoh@gmail.com wrote:
Hi,
From what you are saying, it sounds like all these issues are caused by using an in-memory db… Are you sure that you are connecting to a database, and that the user has the proper privileges?
Try to look for dhis2 home references in the log… it should be early in the startup
–
Morten
On Wed, Dec 18, 2013 at 10:09 AM, Edwin Mulwa eddiemu@gmail.com wrote:
Hi,
I have some major problems when I am trying to import meta-data and data from one DHIS2 instance to another. These are as follows:
Issue 1:
I have a DHIS2 installation running on a VMWare Virtual Machine. Using the Meta-Data Export/Import features in DHIS2, I have imported data from another DHIS2 instance running on a laptop into the VM “successfully”. “Successfully” in quotes because the log says it was imported successfully. However, when I shut down the VM and start it again, all the imported data is missing from the VM. That is, it is as if I did not import any data. In the data import screen, Dry Run option is set to “No”. I did take a snapshot of the VM immediately after importation and reverted to the snapshot after firing up the VM but nothing doing. Any ideas on why the data is not persisting on the VM?
Note: Nothing I am doing on the VM persists! For example, I looked at the default user I have been using to access DHIS2 (admin:district) and added all the roles to the user. These additions to the user account were not reflected after restarting the VM.
Important Note: I have noticed, by looking at the logs, that every time the DHIS2 instance starts up, it is attempting to run alot of table altering SQL commands, most of which are failing. I suspect that it is these commands that are interfering with my data. If you would like to see a copy of the log file, I will be happy to provide it.
Issue 2:
After importing the data, I log out of DHIS2 and try logging in using a different user whose info has been imported into the VM. This does not work. This is BEFORE I even shut down the machine. When I log back in using the default admin user, I can see the user accounts in the system.
Note: I have seen from an old posting on the forum that importation of users was disabled. I don’t know if this is still the case and therefore why I am having this problem?
Issue 3:
After importing meta-data, I immediately tried to import DATA from the other instance. Data import fails with a message that it was unable to clear the BatchHandler.
I still cannot log in using an imported user. I suspect is it for the
reason I mentioned in my original post i.e. importation of users was
disabled?
Did you check to see that the users exists in the system? we do support
import, but we null out the passwords.. for security reasons. So you will
need to re-set the passwords.
The other remaining problem is the importation of XML data. It is still
giving a "Failed to flush BatchHandler" error when I try to import the
data. I have attached the relevant section of the log file so that you
check and see if you can make anything of it.
I'm not sure about that one.. it says duplicate value, but I guess that
should have been treated as an update.. could you file a bug report for it?
···
--
Morten
Thanks alot Morten!
Edwin
On Wed, Dec 18, 2013 at 1:23 AM, Morten Olav Hansen <mortenoh@gmail.com>wrote:
Hi,
From what you are saying, it sounds like all these issues are caused by
using an in-memory db.. Are you sure that you are connecting to a database,
and that the user has the proper privileges?
Try to look for dhis2 home references in the log.. it should be early in
the startup
--
Morten
On Wed, Dec 18, 2013 at 10:09 AM, Edwin Mulwa <eddiemu@gmail.com> wrote:
Hi,
I have some major problems when I am trying to import meta-data and data
from one DHIS2 instance to another. These are as follows:
Issue 1:
I have a DHIS2 installation running on a VMWare Virtual Machine. Using
the Meta-Data Export/Import features in DHIS2, I have imported data from
another DHIS2 instance running on a laptop into the VM "successfully".
"Successfully" in quotes because the log says it was imported successfully.
However, when I shut down the VM and start it again, all the imported data
is missing from the VM. That is, it is as if I did not import any data. In
the data import screen, Dry Run option is set to "No". I did take a
snapshot of the VM immediately after importation and reverted to the
snapshot after firing up the VM but nothing doing. Any ideas on why the
data is not persisting on the VM?
Note: Nothing I am doing on the VM persists! For example, I looked at
the default user I have been using to access DHIS2 (admin:district) and
added all the roles to the user. These additions to the user account were
not reflected after restarting the VM.
Important Note: I have noticed, by looking at the logs, that every time
the DHIS2 instance starts up, it is attempting to run alot of table
altering SQL commands, most of which are failing. I suspect that it is
these commands that are interfering with my data. If you would like to see
a copy of the log file, I will be happy to provide it.
Issue 2:
After importing the data, I log out of DHIS2 and try logging in using a
different user whose info has been imported into the VM. This does not
work. This is BEFORE I even shut down the machine. When I log back in using
the default admin user, I can see the user accounts in the system.
Note: I have seen from an old posting on the forum that importation of
users was disabled. I don't know if this is still the case and therefore
why I am having this problem?
Issue 3:
After importing meta-data, I immediately tried to import DATA from the
other instance. Data import fails with a message that it was unable to
clear the BatchHandler.
I have seen this before and I guess it is not ideal behaviour. When there are two datavalues with the same Key (dataelementid, periodid, sourceid, categoryoptioncomboid) then the importer can’t really know which to take as the real one, so instead it just crashes out a little unceremoniously. We should handle this better. But meanwhile if you cleanup that data before you import you should be fine.
Thank you! The in-memory db was the cause of one of my problems. So Issue 1 is resolved!
I still cannot log in using an imported user. I suspect is it for the reason I mentioned in my original post i.e. importation of users was disabled?
The other remaining problem is the importation of XML data. It is still giving a “Failed to flush BatchHandler” error when I try to import the data. I have attached the relevant section of the log file so that you check and see if you can make anything of it.
On Wed, Dec 18, 2013 at 1:23 AM, Morten Olav Hansen mortenoh@gmail.com wrote:
Hi,
From what you are saying, it sounds like all these issues are caused by using an in-memory db… Are you sure that you are connecting to a database, and that the user has the proper privileges?
Try to look for dhis2 home references in the log… it should be early in the startup
–
Morten
On Wed, Dec 18, 2013 at 10:09 AM, Edwin Mulwa eddiemu@gmail.com wrote:
Hi,
I have some major problems when I am trying to import meta-data and data from one DHIS2 instance to another. These are as follows:
Issue 1:
I have a DHIS2 installation running on a VMWare Virtual Machine. Using the Meta-Data Export/Import features in DHIS2, I have imported data from another DHIS2 instance running on a laptop into the VM “successfully”. “Successfully” in quotes because the log says it was imported successfully. However, when I shut down the VM and start it again, all the imported data is missing from the VM. That is, it is as if I did not import any data. In the data import screen, Dry Run option is set to “No”. I did take a snapshot of the VM immediately after importation and reverted to the snapshot after firing up the VM but nothing doing. Any ideas on why the data is not persisting on the VM?
Note: Nothing I am doing on the VM persists! For example, I looked at the default user I have been using to access DHIS2 (admin:district) and added all the roles to the user. These additions to the user account were not reflected after restarting the VM.
Important Note: I have noticed, by looking at the logs, that every time the DHIS2 instance starts up, it is attempting to run alot of table altering SQL commands, most of which are failing. I suspect that it is these commands that are interfering with my data. If you would like to see a copy of the log file, I will be happy to provide it.
Issue 2:
After importing the data, I log out of DHIS2 and try logging in using a different user whose info has been imported into the VM. This does not work. This is BEFORE I even shut down the machine. When I log back in using the default admin user, I can see the user accounts in the system.
Note: I have seen from an old posting on the forum that importation of users was disabled. I don’t know if this is still the case and therefore why I am having this problem?
Issue 3:
After importing meta-data, I immediately tried to import DATA from the other instance. Data import fails with a message that it was unable to clear the BatchHandler.
I still cannot log in using an imported user. I suspect is it for the
reason I mentioned in my original post i.e. importation of users was
disabled?
Did you check to see that the users exists in the system? we do support
import, but we null out the passwords.. for security reasons. So you will
need to re-set the passwords.
The other remaining problem is the importation of XML data. It is still
giving a "Failed to flush BatchHandler" error when I try to import the
data. I have attached the relevant section of the log file so that you
check and see if you can make anything of it.
I'm not sure about that one.. it says duplicate value, but I guess that
should have been treated as an update.. could you file a bug report for it?
Yes its definitely a bug or at least an ugly behaviour. I should have
reported it earlier - we saw the same importing unclean data from iHRIS in
Rwanda. Once we cleaned the data up all was fine. Updates only happen
when these are part of separate batches. Duplicates in the same import
batch get stuck in the throat.
···
On 18 December 2013 13:30, Morten Olav Hansen <mortenoh@gmail.com> wrote:
--
Morten
Thanks alot Morten!
Edwin
On Wed, Dec 18, 2013 at 1:23 AM, Morten Olav Hansen <mortenoh@gmail.com>wrote:
Hi,
From what you are saying, it sounds like all these issues are caused by
using an in-memory db.. Are you sure that you are connecting to a database,
and that the user has the proper privileges?
Try to look for dhis2 home references in the log.. it should be early in
the startup
--
Morten
On Wed, Dec 18, 2013 at 10:09 AM, Edwin Mulwa <eddiemu@gmail.com> wrote:
Hi,
I have some major problems when I am trying to import meta-data and
data from one DHIS2 instance to another. These are as follows:
Issue 1:
I have a DHIS2 installation running on a VMWare Virtual Machine. Using
the Meta-Data Export/Import features in DHIS2, I have imported data from
another DHIS2 instance running on a laptop into the VM "successfully".
"Successfully" in quotes because the log says it was imported successfully.
However, when I shut down the VM and start it again, all the imported data
is missing from the VM. That is, it is as if I did not import any data. In
the data import screen, Dry Run option is set to "No". I did take a
snapshot of the VM immediately after importation and reverted to the
snapshot after firing up the VM but nothing doing. Any ideas on why the
data is not persisting on the VM?
Note: Nothing I am doing on the VM persists! For example, I looked at
the default user I have been using to access DHIS2 (admin:district) and
added all the roles to the user. These additions to the user account were
not reflected after restarting the VM.
Important Note: I have noticed, by looking at the logs, that every time
the DHIS2 instance starts up, it is attempting to run alot of table
altering SQL commands, most of which are failing. I suspect that it is
these commands that are interfering with my data. If you would like to see
a copy of the log file, I will be happy to provide it.
Issue 2:
After importing the data, I log out of DHIS2 and try logging in using a
different user whose info has been imported into the VM. This does not
work. This is BEFORE I even shut down the machine. When I log back in using
the default admin user, I can see the user accounts in the system.
Note: I have seen from an old posting on the forum that importation of
users was disabled. I don't know if this is still the case and therefore
why I am having this problem?
Issue 3:
After importing meta-data, I immediately tried to import DATA from the
other instance. Data import fails with a message that it was unable to
clear the BatchHandler.
I will take a look at the data and see where the duplicates may be. The data was generated using Data-Export feature in DHIS so there is certainly a problem with this feature. When I was importing, I had selected New and Updates. The “Updates” bit of it should have handled the duplicates. To test this, when I selected “Updates Only”, the import went through successfully.
Morten, as for the user, yes I am sure the user exists in the database. I will reset password and see what happens.
Yes its definitely a bug or at least an ugly behaviour. I should have reported it earlier - we saw the same importing unclean data from iHRIS in Rwanda. Once we cleaned the data up all was fine. Updates only happen when these are part of separate batches. Duplicates in the same import batch get stuck in the throat.
I still cannot log in using an imported user. I suspect is it for the reason I mentioned in my original post i.e. importation of users was disabled?
Did you check to see that the users exists in the system? we do support import, but we null out the passwords… for security reasons. So you will need to re-set the passwords.
The other remaining problem is the importation of XML data. It is still giving a “Failed to flush BatchHandler” error when I try to import the data. I have attached the relevant section of the log file so that you check and see if you can make anything of it.
I’m not sure about that one… it says duplicate value, but I guess that should have been treated as an update… could you file a bug report for it?
–
Morten
Thanks alot Morten!
Edwin
On Wed, Dec 18, 2013 at 1:23 AM, Morten Olav Hansen mortenoh@gmail.com wrote:
Hi,
From what you are saying, it sounds like all these issues are caused by using an in-memory db… Are you sure that you are connecting to a database, and that the user has the proper privileges?
Try to look for dhis2 home references in the log… it should be early in the startup
–
Morten
On Wed, Dec 18, 2013 at 10:09 AM, Edwin Mulwa eddiemu@gmail.com wrote:
Hi,
I have some major problems when I am trying to import meta-data and data from one DHIS2 instance to another. These are as follows:
Issue 1:
I have a DHIS2 installation running on a VMWare Virtual Machine. Using the Meta-Data Export/Import features in DHIS2, I have imported data from another DHIS2 instance running on a laptop into the VM “successfully”. “Successfully” in quotes because the log says it was imported successfully. However, when I shut down the VM and start it again, all the imported data is missing from the VM. That is, it is as if I did not import any data. In the data import screen, Dry Run option is set to “No”. I did take a snapshot of the VM immediately after importation and reverted to the snapshot after firing up the VM but nothing doing. Any ideas on why the data is not persisting on the VM?
Note: Nothing I am doing on the VM persists! For example, I looked at the default user I have been using to access DHIS2 (admin:district) and added all the roles to the user. These additions to the user account were not reflected after restarting the VM.
Important Note: I have noticed, by looking at the logs, that every time the DHIS2 instance starts up, it is attempting to run alot of table altering SQL commands, most of which are failing. I suspect that it is these commands that are interfering with my data. If you would like to see a copy of the log file, I will be happy to provide it.
Issue 2:
After importing the data, I log out of DHIS2 and try logging in using a different user whose info has been imported into the VM. This does not work. This is BEFORE I even shut down the machine. When I log back in using the default admin user, I can see the user accounts in the system.
Note: I have seen from an old posting on the forum that importation of users was disabled. I don’t know if this is still the case and therefore why I am having this problem?
Issue 3:
After importing meta-data, I immediately tried to import DATA from the other instance. Data import fails with a message that it was unable to clear the BatchHandler.
