java.lang.OutOfMemoryError: unable to create new native thread - Weblogic

This is a discussion on java.lang.OutOfMemoryError: unable to create new native thread - Weblogic ; Hi, I have deployed my app on WLS 7.1 sp4. This is Sun Solaris box. The app works fine. But after some time I get "OutOfMemoryError". At the same time, I have not yet got this error on WLS 7.1 ...

+ Reply to Thread
Results 1 to 13 of 13

Thread: java.lang.OutOfMemoryError: unable to create new native thread

  1. java.lang.OutOfMemoryError: unable to create new native thread


    Hi,

    I have deployed my app on WLS 7.1 sp4. This is Sun Solaris box. The app works
    fine. But after some time I get "OutOfMemoryError".

    At the same time, I have not yet got this error on WLS 7.1 sp4 on Windows 2000
    box.

    Any ideas????


    Thanks.

    Peter


    Start of Trace
    ---------------

    #### '8' for queue: 'default'> <> <010051> invocation from home: com.infinity.imaging.afo.business.ejb.session.stat eless.StartupFacadeBean_lwnr6f_HomeImpl@554af0
    threw exception: java.lang.OutOfMemoryError: unable to create new native thread>

    java.lang.OutOfMemoryError: unable to create new native thread
    at java.lang.Thread.start(Native Method)
    at org.apache.log4j.xml.DOMConfigurator.configureAndW atch(DOMConfigurator.java:576)
    at com.infinity.imaging.afo.integration.dao.AFOBaseDA O.(AFOBaseDAO.java:27)
    at com.infinity.imaging.afo.integration.dao.DAOFactor y.(DAOFactory.java:28)
    at com.infinity.imaging.afo.integration.dao.OracleDAO Factory.(OracleDAOFactory.java:38)
    at com.infinity.imaging.afo.integration.dao.DAOFactor y.getDAOFactory(DAOFactory.java:34)
    at com.infinity.imaging.afo.business.ejb.session.stat eless.StartupFacadeBean.getDatabaseName(StartupFac adeBean.java:97)
    at com.infinity.imaging.afo.business.ejb.session.stat eless.StartupFacadeBean_lwnr6f_EOImpl.getDatabaseN ame(StartupFacadeBean_lwnr6f_EOImpl.java:46)
    at com.infinity.imaging.afo.business.ejb.session.stat eless.StartupFacadeBean_lwnr6f_EOImpl_WLSkel.invok e(Unknown
    Source)
    at weblogic.rmi.internal.ServerRequest.sendReceive(Se rverRequest.java:159)
    at weblogic.rmi.cluster.ReplicaAwareRemoteRef.invoke( ReplicaAwareRemoteRef.java:263)
    at weblogic.rmi.cluster.ReplicaAwareRemoteRef.invoke( ReplicaAwareRemoteRef.java:230)
    at com.infinity.imaging.afo.business.ejb.session.stat eless.StartupFacadeBean_lwnr6f_EOImpl_WLStub.getDa tabaseName(Unknown
    Source)
    at com.infinity.imaging.afo.business.service.StartupD elegate.getDatabaseName(StartupDelegate.java:102)
    at com.infinity.imaging.afo.presentation.action.DoSta rtAction.execute(DoStartAction.java:44)
    at org.apache.struts.action.RequestProcessor.processA ctionPerform(RequestProcessor.java:484)
    at org.apache.struts.action.RequestProcessor.process( RequestProcessor.java:274)
    at org.apache.struts.action.ActionServlet.process(Act ionServlet.java:1482)
    at org.apache.struts.action.ActionServlet.doGet(Actio nServlet.java:507)
    at javax.servlet.http.HttpServlet.service(HttpServlet .java:740)
    at javax.servlet.http.HttpServlet.service(HttpServlet .java:853)
    at weblogic.servlet.internal.ServletStubImpl$ServletI nvocationAction.run(ServletStubImpl.java:1075)
    at weblogic.servlet.internal.ServletStubImpl.invokeSe rvlet(ServletStubImpl.java:418)
    at weblogic.servlet.internal.ServletStubImpl.invokeSe rvlet(ServletStubImpl.java:306)
    at weblogic.servlet.internal.WebAppServletContext$Ser vletInvocationAction.run(WebAppServletContext.java :5517)
    at weblogic.security.service.SecurityServiceManager.r unAs(SecurityServiceManager.java:685)
    at weblogic.servlet.internal.WebAppServletContext.inv okeServlet(WebAppServletContext.java:3156)
    at weblogic.servlet.internal.ServletRequestImpl.execu te(ServletRequestImpl.java:2506)
    at weblogic.kernel.ExecuteThread.execute(ExecuteThrea d.java:234)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.ja va:210)

  2. Re: java.lang.OutOfMemoryError: unable to create new native thread

    Its probably because your solaris system has less physical memory available.
    Check if other applications are using up the memory. On solaris its possible
    to switch to a newer alternate thread model to get better performance check
    solaris documentation.




    "Peter Vennel" wrote in message
    news:40f3fadc$1@mktnews1...
    >
    > Hi,
    >
    > I have deployed my app on WLS 7.1 sp4. This is Sun Solaris box. The app

    works
    > fine. But after some time I get "OutOfMemoryError".
    >
    > At the same time, I have not yet got this error on WLS 7.1 sp4 on Windows

    2000
    > box.
    >
    > Any ideas????
    >
    >
    > Thanks.
    >
    > Peter
    >
    >
    > Start of Trace
    > ---------------
    >
    > ####


    > '8' for queue: 'default'> <> <010051>
    during
    > invocation from home:

    com.infinity.imaging.afo.business.ejb.session.stat eless.StartupFacadeBean_lw
    nr6f_HomeImpl@554af0
    > threw exception: java.lang.OutOfMemoryError: unable to create new native

    thread>
    >
    > java.lang.OutOfMemoryError: unable to create new native thread
    > at java.lang.Thread.start(Native Method)
    > at

    org.apache.log4j.xml.DOMConfigurator.configureAndW atch(DOMConfigurator.java:
    576)
    > at

    com.infinity.imaging.afo.integration.dao.AFOBaseDA O.(AFOBaseDAO.java:2
    7)
    > at

    com.infinity.imaging.afo.integration.dao.DAOFactor y.(DAOFactory.java:2
    8)
    > at

    com.infinity.imaging.afo.integration.dao.OracleDAO Factory.(OracleDAOFa
    ctory.java:38)
    > at

    com.infinity.imaging.afo.integration.dao.DAOFactor y.getDAOFactory(DAOFactory
    ..java:34)
    > at

    com.infinity.imaging.afo.business.ejb.session.stat eless.StartupFacadeBean.ge
    tDatabaseName(StartupFacadeBean.java:97)
    > at

    com.infinity.imaging.afo.business.ejb.session.stat eless.StartupFacadeBean_lw
    nr6f_EOImpl.getDatabaseName(StartupFacadeBean_lwnr 6f_EOImpl.java:46)
    > at

    com.infinity.imaging.afo.business.ejb.session.stat eless.StartupFacadeBean_lw
    nr6f_EOImpl_WLSkel.invoke(Unknown
    > Source)
    > at weblogic.rmi.internal.ServerRequest.sendReceive(Se rverRequest.java:159)
    > at

    weblogic.rmi.cluster.ReplicaAwareRemoteRef.invoke( ReplicaAwareRemoteRef.java
    :263)
    > at

    weblogic.rmi.cluster.ReplicaAwareRemoteRef.invoke( ReplicaAwareRemoteRef.java
    :230)
    > at

    com.infinity.imaging.afo.business.ejb.session.stat eless.StartupFacadeBean_lw
    nr6f_EOImpl_WLStub.getDatabaseName(Unknown
    > Source)
    > at

    com.infinity.imaging.afo.business.service.StartupD elegate.getDatabaseName(St
    artupDelegate.java:102)
    > at

    com.infinity.imaging.afo.presentation.action.DoSta rtAction.execute(DoStartAc
    tion.java:44)
    > at

    org.apache.struts.action.RequestProcessor.processA ctionPerform(RequestProces
    sor.java:484)
    > at

    org.apache.struts.action.RequestProcessor.process( RequestProcessor.java:274)
    > at org.apache.struts.action.ActionServlet.process(Act ionServlet.java:1482)
    > at org.apache.struts.action.ActionServlet.doGet(Actio nServlet.java:507)
    > at javax.servlet.http.HttpServlet.service(HttpServlet .java:740)
    > at javax.servlet.http.HttpServlet.service(HttpServlet .java:853)
    > at

    weblogic.servlet.internal.ServletStubImpl$ServletI nvocationAction.run(Servle
    tStubImpl.java:1075)
    > at

    weblogic.servlet.internal.ServletStubImpl.invokeSe rvlet(ServletStubImpl.java
    :418)
    > at

    weblogic.servlet.internal.ServletStubImpl.invokeSe rvlet(ServletStubImpl.java
    :306)
    > at

    weblogic.servlet.internal.WebAppServletContext$Ser vletInvocationAction.run(W
    ebAppServletContext.java:5517)
    > at

    weblogic.security.service.SecurityServiceManager.r unAs(SecurityServiceManage
    r.java:685)
    > at

    weblogic.servlet.internal.WebAppServletContext.inv okeServlet(WebAppServletCo
    ntext.java:3156)
    > at

    weblogic.servlet.internal.ServletRequestImpl.execu te(ServletRequestImpl.java
    :2506)
    > at weblogic.kernel.ExecuteThread.execute(ExecuteThrea d.java:234)
    > at weblogic.kernel.ExecuteThread.run(ExecuteThread.ja va:210)




  3. Re: java.lang.OutOfMemoryError: unable to create new native thread

    "Peter Vennel" wrote in message news:40f3fadc$1@mktnews1...
    > I have deployed my app on WLS 7.1 sp4. This is Sun Solaris box. The app works
    > fine. But after some time I get "OutOfMemoryError".
    >
    > At the same time, I have not yet got this error on WLS 7.1 sp4 on Windows 2000
    > box.


    The likely cause of this is that your application creates a lot of threads
    thus hitting limit of native memory available for thread stacks. The easiest
    way to fix it is too reduce memory allocated per thread stack. For solaris
    it's 512K for 1.3.1. To reduce it, add

    -Xss384k

    to JVM startup options.

    Also, get a thread dump when the app is running under load and
    try to find out if there are "run-away" threads.

    Hope this helps.

    Regards,

    Slava Imeshev



  4. Re: java.lang.OutOfMemoryError: unable to create new native thread


    Thanks Sree.

    As you correctly mentioned, our solaris system has only 512 MB of memory. But
    right now, only 1-2 users are using the app on the server. All we have is 4-5
    stateless EJBs running. I wonder what might happen on production when we have
    25-50 users.

    I tried reducing the java heap, thereby increasing the native heap. Now the max
    threads that I could run changed from 3400 to 4800.

    I have asked our Unix guy to look into the 'alternate thread model' you suggested.

    I am still very skeptical of this when we move to production box.


    "sree bodapati" wrote:
    >Its probably because your solaris system has less physical memory available.
    >Check if other applications are using up the memory. On solaris its possible
    >to switch to a newer alternate thread model to get better performance
    >check
    >solaris documentation.
    >



  5. Re: java.lang.OutOfMemoryError: unable to create new native thread

    "Peter Vennel" wrote in message news:40f429c6$1@mktnews1...
    > As you correctly mentioned, our solaris system has only 512 MB of memory. But
    > right now, only 1-2 users are using the app on the server. All we have is 4-5
    > stateless EJBs running. I wonder what might happen on production when we have
    > 25-50 users.
    >
    > I tried reducing the java heap, thereby increasing the native heap. Now the max
    > threads that I could run changed from 3400 to 4800.
    >
    > I have asked our Unix guy to look into the 'alternate thread model' you suggested.
    >
    > I am still very skeptical of this when we move to production box.


    You have to get a thread dump and see if/where/why there are too many threads.
    Without it you can only be guessing. You get a thread dump by sending

    kill -3

    Also, as I mentioned, reduce the stack size. This alone may fix the problem.

    Slava

    >
    >
    > "sree bodapati" wrote:
    > >Its probably because your solaris system has less physical memory available.
    > >Check if other applications are using up the memory. On solaris its possible
    > >to switch to a newer alternate thread model to get better performance
    > >check
    > >solaris documentation.
    > >

    >




  6. Re: java.lang.OutOfMemoryError: unable to create new native thread


    Thanks Slava.

    I want to know why the threads are not getting recycled. Is there a bug in Weblogic
    or a flaw in my architecture??

    I am using couple of Stateless Session Beans which are called by Business Delegate

    using EJBHomeFactory method.


    I have done the following in my EJB design.

    1. Since I am using Stateless SB, I am caching the EJBHome.
    2. Implementing Business Delegate as Singleton.
    3. Implementing EJBHomeFactory as Singleton.
    4. Not calling remove() on EJB as it is handled by Container.


    I am not sure where I am going wrong. But I noticed that every call to the EJB
    (BusinessDelegate --> EJBHomeFactory ---> SessionFacade)
    spawns a new thread on server (which does not ever get recycled). Is this normal?
    Eventually I come to a state
    where the server runs out of memory and cannot create any new native thread.

    Why are the threads not recycled???

    Can someone please advise where the problem lies...

    Thanks.

    Peter


    "Slava Imeshev" wrote:
    >"Peter Vennel" wrote in message news:40f3fadc$1@mktnews1...
    >> I have deployed my app on WLS 7.1 sp4. This is Sun Solaris box. The

    >app works
    >> fine. But after some time I get "OutOfMemoryError".
    >>
    >> At the same time, I have not yet got this error on WLS 7.1 sp4 on Windows

    >2000
    >> box.

    >
    >The likely cause of this is that your application creates a lot of threads
    >thus hitting limit of native memory available for thread stacks. The
    >easiest
    >way to fix it is too reduce memory allocated per thread stack. For solaris
    >it's 512K for 1.3.1. To reduce it, add
    >
    > -Xss384k
    >
    >to JVM startup options.
    >
    >Also, get a thread dump when the app is running under load and
    >try to find out if there are "run-away" threads.
    >
    >Hope this helps.
    >
    >Regards,
    >
    >Slava Imeshev
    >
    >



  7. Re: java.lang.OutOfMemoryError: unable to create new native thread

    "Peter Vennel" wrote in message news:40f4538c$1@mktnews1...
    > Thanks Slava.
    >
    > I want to know why the threads are not getting recycled. Is there a bug in Weblogic
    > or a flaw in my architecture??


    I don't think it's a bug in weblogic. weblogic is a mature product used by
    a bunch of people, and from my experience this kind of problems shows
    up either when 3rdparty APIs are misused or due to engineering mistakes.

    I cannot tell if it's flaw in your architecture as I don't know much about your app.

    You need to get a thread dump and post it here in the newsgroup - we'll try to help.

    > I am not sure where I am going wrong. But I noticed that every call to the EJB
    > (BusinessDelegate --> EJBHomeFactory ---> SessionFacade)
    > spawns a new thread on server (which does not ever get recycled). Is this normal?
    > Eventually I come to a state
    > where the server runs out of memory and cannot create any new native thread.
    >
    > Why are the threads not recycled???


    I have to repeat it again - get the thread dump and post it here.

    > Can someone please advise where the problem lies...


    Getting the tread dump is the first step in fixing this type of problems.

    Slava

    > "Slava Imeshev" wrote:
    > >"Peter Vennel" wrote in message news:40f3fadc$1@mktnews1...
    > >> I have deployed my app on WLS 7.1 sp4. This is Sun Solaris box. The

    > >app works
    > >> fine. But after some time I get "OutOfMemoryError".
    > >>
    > >> At the same time, I have not yet got this error on WLS 7.1 sp4 on Windows

    > >2000
    > >> box.

    > >
    > >The likely cause of this is that your application creates a lot of threads
    > >thus hitting limit of native memory available for thread stacks. The
    > >easiest
    > >way to fix it is too reduce memory allocated per thread stack. For solaris
    > >it's 512K for 1.3.1. To reduce it, add
    > >
    > > -Xss384k
    > >
    > >to JVM startup options.
    > >
    > >Also, get a thread dump when the app is running under load and
    > >try to find out if there are "run-away" threads.
    > >
    > >Hope this helps.
    > >
    > >Regards,
    > >
    > >Slava Imeshev
    > >
    > >

    >




  8. Re: java.lang.OutOfMemoryError: unable to create new native thread

    "Peter Vennel" wrote in message news:40f46669$1@mktnews1...
    > To create coredump in WLS 5.1, you had to disable JIT compiler by addding -nojit
    > to command line. Can you tell me as to what is the equivalent of that for WLS
    > 7.0.4. For some reason, I could not locate this.


    I believe there shouldn't be anything special done for 7.x.


    Slava





  9. issue resolved


    Finally I could resolve the issue.

    I am using DOMConfigurator.configureAndWatch() for dynamically reading the log4j
    xml file. What I noticed was that this was spawning a new thread and this thread
    was never recycled. I need to look into this later. But for now, I masked theline
    containing this and everything works fine.

    Thanks Slava & Sree for all your help.

  10. Re: issue resolved

    "Peter Vennel" wrote in message news:40f58a54$1@mktnews1...
    > Finally I could resolve the issue.
    >
    > I am using DOMConfigurator.configureAndWatch() for dynamically reading the log4j
    > xml file. What I noticed was that this was spawning a new thread and this thread
    > was never recycled. I need to look into this later. But for now, I masked theline
    > containing this and everything works fine.


    configureAndWatch should be called once per application startup, that it.

    > Thanks Slava & Sree for all your help.


    Glad to help.

    Regards,

    Slava Imeshev



  11. Re: issue resolved


    Hi Slava,

    I made a blunder of having it all classes that require logging.

    But now I reliazed that I need to have a startup servlet and add this to it.
    But now my problem is that I have an EJB module which is separate from the Web
    Module. So where do I specify configureAndWatch() ?

    Please advise.

    Thanks.
    Peter

    "Slava Imeshev" wrote:
    >
    >configureAndWatch should be called once per application startup, that it.




  12. Re: issue resolved

    "Peter Vennel" wrote in message news:40f6758b$1@mktnews1...
    > I made a blunder of having it all classes that require logging.
    >
    > But now I reliazed that I need to have a startup servlet and add this to it.
    > But now my problem is that I have an EJB module which is separate from the Web
    > Module. So where do I specify configureAndWatch() ?


    weblogic provides server lifecycle interfaces:

    http://e-docs.bea.com/wls/docs81/Con..._shutdown.html


    > Please advise.


    Also, check
    http://www.viewtier.com/newsgroups/t...dID=4&tstart=0


    Regards,

    Slava Imeshev



  13. Re: issue resolved


    Thanks Slava. You just made my day ;->


    "Slava Imeshev" wrote:
    >"Peter Vennel" wrote in message news:40f6758b$1@mktnews1...
    >> I made a blunder of having it all classes that require logging.
    >>
    >> But now I reliazed that I need to have a startup servlet and add this

    >to it.
    >> But now my problem is that I have an EJB module which is separate from

    >the Web
    >> Module. So where do I specify configureAndWatch() ?

    >
    >weblogic provides server lifecycle interfaces:
    >
    >http://e-docs.bea.com/wls/docs81/Con..._shutdown.html
    >
    >
    >> Please advise.

    >
    >Also, check
    > http://www.viewtier.com/newsgroups/t...dID=4&tstart=0
    >
    >
    >Regards,
    >
    >Slava Imeshev
    >
    >



+ Reply to Thread