obiz 13 thread deadlock 5000 concurrent users monitor org.base.start.Classpath$NativeLibClassLoader

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

obiz 13 thread deadlock 5000 concurrent users monitor org.base.start.Classpath$NativeLibClassLoader

Justin Robinson-2
I'm profiling a downstream project version 10, I recently
replaced/refactored/bridged the framework components with obiz 13 and some
obiz 14 (just the transaction related in entity, to use dbcp2 etc.

Anyway the monitor deadlocks I saw with initial testing on the version 10
based multitenant erp system are all gone and in there place is this one
monitor. In jmeter tests of 5000 users spread accross 200 tenants I'm
getting 100% failure and all threads are blocked and time-out as a result

5 000 http-bio-0.0.0-8080-exec-xxx deadlock on monitor
org.base.start.Classpath$NativeLibClassLoader
Reply | Threaded
Open this post in threaded view
|

Re: obiz 13 thread deadlock 5000 concurrent users monitor org.base.start.Classpath$NativeLibClassLoader

taher
Hi Justin,

What do mean by project version 10? Do you mean OFBiz 10?

Also do you have logs to show the progression towards dead locks? This
issue seems extremely technical with backported changes from two releases
back an old unsupported release. It would almost be impossible to help you
out without some context and background details.

FYI the class below is an implementation of OFBiz specific classloader that
adds stuff to the classpath. If you made customizations to the classpath
then that might be a reason.

HTH

Taher Alkhateeb
On Jul 9, 2016 2:20 PM, "Justin Robinson" <
[hidden email]> wrote:

> I'm profiling a downstream project version 10, I recently
> replaced/refactored/bridged the framework components with obiz 13 and some
> obiz 14 (just the transaction related in entity, to use dbcp2 etc.
>
> Anyway the monitor deadlocks I saw with initial testing on the version 10
> based multitenant erp system are all gone and in there place is this one
> monitor. In jmeter tests of 5000 users spread accross 200 tenants I'm
> getting 100% failure and all threads are blocked and time-out as a result
>
> 5 000 http-bio-0.0.0-8080-exec-xxx deadlock on monitor
> org.base.start.Classpath$NativeLibClassLoader
>
Reply | Threaded
Open this post in threaded view
|

Re: obiz 13 thread deadlock 5000 concurrent users monitor org.base.start.Classpath$NativeLibClassLoader

Justin Robinson-2
This commercial erp is highly customised downstream project of ofbiz 10.04,
OT actually. I profiled the app, identified changes in the framework
changes and ensured I worked them back into the code, a bridged port if you
will. I looked at the solution ofbiz applied to these very same threading
issues I found. But framework/start was taken directly from ofbiz 13,
nothing needed to be changed because this part of framework was not
customized. Most of the other customization to framework are superficial.

On Sat, Jul 9, 2016 at 6:19 PM, Taher Alkhateeb <[hidden email]>
wrote:

> Hi Justin,
>
> What do mean by project version 10? Do you mean OFBiz 10?
>
> Also do you have logs to show the progression towards dead locks? This
> issue seems extremely technical with backported changes from two releases
> back an old unsupported release. It would almost be impossible to help you
> out without some context and background details.
>
> FYI the class below is an implementation of OFBiz specific classloader that
> adds stuff to the classpath. If you made customizations to the classpath
> then that might be a reason.
>
> HTH
>
> Taher Alkhateeb
> On Jul 9, 2016 2:20 PM, "Justin Robinson" <
> [hidden email]> wrote:
>
> > I'm profiling a downstream project version 10, I recently
> > replaced/refactored/bridged the framework components with obiz 13 and
> some
> > obiz 14 (just the transaction related in entity, to use dbcp2 etc.
> >
> > Anyway the monitor deadlocks I saw with initial testing on the version 10
> > based multitenant erp system are all gone and in there place is this one
> > monitor. In jmeter tests of 5000 users spread accross 200 tenants I'm
> > getting 100% failure and all threads are blocked and time-out as a result
> >
> > 5 000 http-bio-0.0.0-8080-exec-xxx deadlock on monitor
> > org.base.start.Classpath$NativeLibClassLoader
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: obiz 13 thread deadlock 5000 concurrent users monitor org.base.start.Classpath$NativeLibClassLoader

Justin Robinson-2
the log from those tests is pretty big but will try add important parts to
this thread

On Sat, Jul 9, 2016 at 7:19 PM, Justin Robinson <
[hidden email]> wrote:

> This commercial erp is highly customised downstream project of ofbiz
> 10.04, OT actually. I profiled the app, identified changes in the framework
> changes and ensured I worked them back into the code, a bridged port if you
> will. I looked at the solution ofbiz applied to these very same threading
> issues I found. But framework/start was taken directly from ofbiz 13,
> nothing needed to be changed because this part of framework was not
> customized. Most of the other customization to framework are superficial.
>
> On Sat, Jul 9, 2016 at 6:19 PM, Taher Alkhateeb <
> [hidden email]> wrote:
>
>> Hi Justin,
>>
>> What do mean by project version 10? Do you mean OFBiz 10?
>>
>> Also do you have logs to show the progression towards dead locks? This
>> issue seems extremely technical with backported changes from two releases
>> back an old unsupported release. It would almost be impossible to help you
>> out without some context and background details.
>>
>> FYI the class below is an implementation of OFBiz specific classloader
>> that
>> adds stuff to the classpath. If you made customizations to the classpath
>> then that might be a reason.
>>
>> HTH
>>
>> Taher Alkhateeb
>> On Jul 9, 2016 2:20 PM, "Justin Robinson" <
>> [hidden email]> wrote:
>>
>> > I'm profiling a downstream project version 10, I recently
>> > replaced/refactored/bridged the framework components with obiz 13 and
>> some
>> > obiz 14 (just the transaction related in entity, to use dbcp2 etc.
>> >
>> > Anyway the monitor deadlocks I saw with initial testing on the version
>> 10
>> > based multitenant erp system are all gone and in there place is this one
>> > monitor. In jmeter tests of 5000 users spread accross 200 tenants I'm
>> > getting 100% failure and all threads are blocked and time-out as a
>> result
>> >
>> > 5 000 http-bio-0.0.0-8080-exec-xxx deadlock on monitor
>> > org.base.start.Classpath$NativeLibClassLoader
>> >
>>
>
>