Potential resource leak

Hi all -

   Recently, some custom modifications we were doing for a DHIS2
installation caused a server crash unexpectedly. It turns out that
we were not closing a ResultSet that we were creating directly from a
JdbcTemplate, and this in turn was causing a c3p0 thread resource leak
that would, over time, bring the entire service down. (The endpoint
error was something like "Maximum number of threads (200) created" in
catalina.out...)

   We were able to diagnose the problem and pinpoint the exact place
in our custom source that the problem was at by setting the c3p0
options 'debugUnreturnedConnectionStackTrace' and
'unreturnedConnectionTimeout' to appropriate values - these in concert
cause a stack trace with the location of the created ResultSet to be
emitted as log entries. We fixed our mistake.

   However, during this process, we were able to see another area of
the code which got reported as not closing a resource as well, and on
inspection it seems it was doing the exact thing we were - manually
opening a ResultSet, and not closing it. A partial stack trace from
that event is attached. Indeed, it seems like a number of methods in
the SQL view actions do the same sort of thing...

   I just wanted to bring this to your attention, as if it does
actually represent a potential c3p0 leak it could cause server
instability on potentially any DHIS2 installation on which that
feature is used. Our version of DHIS2 is heavily customized, so it's
not exactly a test case, but I will be downloading stock 2.10 and 2.11
codebases over the next few days to extensively test in this manner.

   Are these known issues? Is my reasoning here sound, or are there
reasons that might this be a false positive?

Best,
-David

stack-trace (2.08 KB)

Hi Dave,

You are right, this code was getting a bit dated and used old-style jdbc connections and result sets. I have re-factored the class your stack trace refers to in trunk to use Spring JdbcTemplate which does resource handling transparently. I did a quick analysis through eclipse and was able to fix a few similar issues too. Many thanks for reporting this.

best regards,

Lars

···

On Thu, Apr 11, 2013 at 10:11 PM, Dave Trombley dave.trombley@gmail.com wrote:

Hi all -

Recently, some custom modifications we were doing for a DHIS2

installation caused a server crash unexpectedly. It turns out that

we were not closing a ResultSet that we were creating directly from a

JdbcTemplate, and this in turn was causing a c3p0 thread resource leak

that would, over time, bring the entire service down. (The endpoint

error was something like “Maximum number of threads (200) created” in

catalina.out…)

We were able to diagnose the problem and pinpoint the exact place

in our custom source that the problem was at by setting the c3p0

options ‘debugUnreturnedConnectionStackTrace’ and

‘unreturnedConnectionTimeout’ to appropriate values - these in concert

cause a stack trace with the location of the created ResultSet to be

emitted as log entries. We fixed our mistake.

However, during this process, we were able to see another area of

the code which got reported as not closing a resource as well, and on

inspection it seems it was doing the exact thing we were - manually

opening a ResultSet, and not closing it. A partial stack trace from

that event is attached. Indeed, it seems like a number of methods in

the SQL view actions do the same sort of thing…

I just wanted to bring this to your attention, as if it does

actually represent a potential c3p0 leak it could cause server

instability on potentially any DHIS2 installation on which that

feature is used. Our version of DHIS2 is heavily customized, so it’s

not exactly a test case, but I will be downloading stock 2.10 and 2.11

codebases over the next few days to extensively test in this manner.

Are these known issues? Is my reasoning here sound, or are there

reasons that might this be a false positive?

Best,

-David


Mailing list: https://launchpad.net/~dhis2-devs

Post to : dhis2-devs@lists.launchpad.net

Unsubscribe : https://launchpad.net/~dhis2-devs

More help : https://help.launchpad.net/ListHelp