Synchronisation issues

[Jacques Le Roux]

Hi,

I'm highly interested by the issues Philippe Mouawad created, notably
https://issues.apache.org/jira/browse/OFBIZ-2186
https://issues.apache.org/jira/browse/OFBIZ-2124

Also, but not directly related to Synchronisation issues in OFBiz
https://issues.apache.org/jira/browse/OFBIZ-2353 (waiting comments from Philippe)
https://issues.apache.org/jira/browse/OFBIZ-2599 (waiting action by DBCP team)

I trust Philippe and his colleagues since I know they worked on the optimisation of a large and successful OFBiz deployment.

David suggested successively 2 patches for OFBIZ-2186 but they did not fix the problem Philippe encountered. However Philippe's
patch did.

Since I'd like to fix these 2 issues in OFBiz (2124 and then 2186) but I have no time to set 2 machines and a load balancer for
these particular problems to test, I googled for "synchronized java performance"
and I found this article http://www.ibm.com/developerworks/java/library/j-jtp04223.html#N1005B by Brian Goetz who also wrote
http://www.javaconcurrencyinpractice.com/ (I did not read the book yet)
I read also in Sun documentation http://java.sun.com/developer/onlineTraining/Programming/JDCBook/perf2.html
"Synchronized methods and objects have until Java 2 always incurred an additional performance hit as the mechanism used to implement
the locking of this code used a global monitor registry which was only single threaded in some areas such as searching for existing
monitors. In the Java 2 release, each thread has a monitor registry and so many of the existing bottlenecks have been removed.
If you have previously used other locking mechanisms because of the performance hit with synchronized methods it is now worthwhile
re-visiting this code and incorporating the new Java 2 streamlined locks."
And also for Java 6 at http://java.sun.com/performance/reference/whitepapers/6_performance.html#2.1

So I would like to commit the patch I slightly modified in OFBIZ-2124

But before that I'd be interested to have some opinions by people who are using load balancers. Notably Brett who recently answered
about this subject on user ML. Because I wonder why Philippe was the 1st to discover theses issues and if others have fixed them in
another way.

Please comment

Thanks

Jacques

[Adam Heath-2]

> But before that I'd be interested to have some opinions by people who
> are using load balancers. Notably Brett who recently answered about this
> subject on user ML. Because I wonder why Philippe was the 1st to
> discover theses issues and if others have fixed them in another way.

Using synchronization can cause deadlocks.  If you do not *fully*
control *all* code that is run under synchronization, then a deadlock
can occur.

If you call some other library, and said library is extensible, such
that it might end up calling *back* into your code, you *will* deadlock.

Non-blocking algorythms don't have this problem, because they retry in
these cases.

[Jacques Le Roux]

From: "Adam Heath" <[hidden email]>
>> But before that I'd be interested to have some opinions by people who
>> are using load balancers. Notably Brett who recently answered about this
>> subject on user ML. Because I wonder why Philippe was the 1st to
>> discover theses issues and if others have fixed them in another way.
>
> Using synchronization can cause deadlocks.  If you do not *fully*
> control *all* code that is run under synchronization, then a deadlock
> can occur.

I agree, synchronization is tricky. BTW, what to you think about r751619 ? Here is a remark about it
https://issues.apache.org/jira/browse/OFBIZ-2186?focusedCommentId=12680105&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12680105

> If you call some other library, and said library is extensible, such
> that it might end up calling *back* into your code, you *will* deadlock.

I agree, but I don't think it concerns this problem (I checked the Call Hierarchy)

> Non-blocking algorythms don't have this problem, because they retry in
> these cases.

Sure, but a problem appeared here, it's well explained at https://issues.apache.org/jira/browse/OFBIZ-2124 and they propose a fix
that works.
It's not the 1st time this kind of problem appears. We have an opportunity to try a fix, this is the pragmatic reason of this
message. What make me worry is that I'm sure there are other sites at least as much "big" as the one Philippe speaks about (but it's
true it's a big one, at least in OFBiz world).

Others opinions ?

Jacques

[Adrian Crum-2]

There is no doubt that there are synchronization issues in the Java code. I run across them all the time and fix them as I find them.

Maybe this is a good time to start a synchronization "Best Practices" thread where we can all offer our insights and suggestions so that poorly synchronized code is easier to notice and correct.

-Adrian


--- On Sat, 11/14/09, Jacques Le Roux <[hidden email]> wrote:

> From: Jacques Le Roux <[hidden email]>
> Subject: Re: Synchronisation issues
> To: [hidden email]
> Date: Saturday, November 14, 2009, 6:58 AM
> From: "Adam Heath" <[hidden email]>
> >> But before that I'd be interested to have some
> opinions by people who
> >> are using load balancers. Notably Brett who
> recently answered about this
> >> subject on user ML. Because I wonder why Philippe
> was the 1st to
> >> discover theses issues and if others have fixed
> them in another way.
> >
> > Using synchronization can cause deadlocks.  If
> you do not *fully*
> > control *all* code that is run under synchronization,
> then a deadlock
> > can occur.
>
> I agree, synchronization is tricky. BTW, what to you think
> about r751619 ? Here is a remark about it
> https://issues.apache.org/jira/browse/OFBIZ-2186?focusedCommentId=12680105&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12680105
>
> > If you call some other library, and said library is
> extensible, such
> > that it might end up calling *back* into your code,
> you *will* deadlock.
>
> I agree, but I don't think it concerns this problem (I
> checked the Call Hierarchy)
>
> > Non-blocking algorythms don't have this problem,
> because they retry in
> > these cases.
>
> Sure, but a problem appeared here, it's well explained at
> https://issues.apache.org/jira/browse/OFBIZ-2124 and
> they propose a fix
> that works.
> It's not the 1st time this kind of problem appears. We have
> an opportunity to try a fix, this is the pragmatic reason of
> this
> message. What make me worry is that I'm sure there are
> other sites at least as much "big" as the one Philippe
> speaks about (but it's
> true it's a big one, at least in OFBiz world).
>
> Others opinions ?
>
> Jacques
>
>
>

[Jacques Le Roux]

Thanks for comment Adrian,

I really think Philippe (and his team) has done a good work. I trust them and I'd like to put that in. I was looking for cons if
any, else I will commit soon...

Thanks

Jacques

From: "Adrian Crum" <[hidden email]>

> There is no doubt that there are synchronization issues in the Java code. I run across them all the time and fix them as I find
> them.
>
> Maybe this is a good time to start a synchronization "Best Practices" thread where we can all offer our insights and suggestions
> so that poorly synchronized code is easier to notice and correct.
>
> -Adrian
>
>
> --- On Sat, 11/14/09, Jacques Le Roux <[hidden email]> wrote:
>
>> From: Jacques Le Roux <[hidden email]>
>> Subject: Re: Synchronisation issues
>> To: [hidden email]
>> Date: Saturday, November 14, 2009, 6:58 AM
>> From: "Adam Heath" <[hidden email]>
>> >> But before that I'd be interested to have some
>> opinions by people who
>> >> are using load balancers. Notably Brett who
>> recently answered about this
>> >> subject on user ML. Because I wonder why Philippe
>> was the 1st to
>> >> discover theses issues and if others have fixed
>> them in another way.
>> >
>> > Using synchronization can cause deadlocks. If
>> you do not *fully*
>> > control *all* code that is run under synchronization,
>> then a deadlock
>> > can occur.
>>
>> I agree, synchronization is tricky. BTW, what to you think
>> about r751619 ? Here is a remark about it
>> https://issues.apache.org/jira/browse/OFBIZ-2186?focusedCommentId=12680105&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12680105
>>
>> > If you call some other library, and said library is
>> extensible, such
>> > that it might end up calling *back* into your code,
>> you *will* deadlock.
>>
>> I agree, but I don't think it concerns this problem (I
>> checked the Call Hierarchy)
>>
>> > Non-blocking algorythms don't have this problem,
>> because they retry in
>> > these cases.
>>
>> Sure, but a problem appeared here, it's well explained at
>> https://issues.apache.org/jira/browse/OFBIZ-2124 and
>> they propose a fix
>> that works.
>> It's not the 1st time this kind of problem appears. We have
>> an opportunity to try a fix, this is the pragmatic reason of
>> this
>> message. What make me worry is that I'm sure there are
>> other sites at least as much "big" as the one Philippe
>> speaks about (but it's
>> true it's a big one, at least in OFBiz world).
>>
>> Others opinions ?
>>
>> Jacques
>>
>>
>>
>
>
>
>

[pierre.gaudin]

Hi  Jacques,

I know that it is not the proof that functions correctly but we work
with Philippe on joint projects and the patchs have been in production
on OFBiz for several months without any problems.

Regards

Pierre

Jacques Le Roux a écrit :

> Thanks for comment Adrian,
>
> I really think Philippe (and his team) has done a good work. I trust
> them and I'd like to put that in. I was looking for cons if
> any, else I will commit soon...
>
> Thanks
>
> Jacques
>
> From: "Adrian Crum" <[hidden email]>
>> There is no doubt that there are synchronization issues in the Java
>> code. I run across them all the time and fix them as I find
>> them.
>>
>> Maybe this is a good time to start a synchronization "Best Practices"
>> thread where we can all offer our insights and suggestions
>> so that poorly synchronized code is easier to notice and correct.
>>
>> -Adrian
>>
>>
>> --- On Sat, 11/14/09, Jacques Le Roux <[hidden email]>
>> wrote:
>>
>>> From: Jacques Le Roux <[hidden email]>
>>> Subject: Re: Synchronisation issues
>>> To: [hidden email]
>>> Date: Saturday, November 14, 2009, 6:58 AM
>>> From: "Adam Heath" <[hidden email]>
>>> >> But before that I'd be interested to have some
>>> opinions by people who
>>> >> are using load balancers. Notably Brett who
>>> recently answered about this
>>> >> subject on user ML. Because I wonder why Philippe
>>> was the 1st to
>>> >> discover theses issues and if others have fixed
>>> them in another way.
>>> >
>>> > Using synchronization can cause deadlocks. If
>>> you do not *fully*
>>> > control *all* code that is run under synchronization,
>>> then a deadlock
>>> > can occur.
>>>
>>> I agree, synchronization is tricky. BTW, what to you think
>>> about r751619 ? Here is a remark about it
>>> https://issues.apache.org/jira/browse/OFBIZ-2186?focusedCommentId=12680105&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12680105 
>>>
>>>
>>> > If you call some other library, and said library is
>>> extensible, such
>>> > that it might end up calling *back* into your code,
>>> you *will* deadlock.
>>>
>>> I agree, but I don't think it concerns this problem (I
>>> checked the Call Hierarchy)
>>>
>>> > Non-blocking algorythms don't have this problem,
>>> because they retry in
>>> > these cases.
>>>
>>> Sure, but a problem appeared here, it's well explained at
>>> https://issues.apache.org/jira/browse/OFBIZ-2124 and
>>> they propose a fix
>>> that works.
>>> It's not the 1st time this kind of problem appears. We have
>>> an opportunity to try a fix, this is the pragmatic reason of
>>> this
>>> message. What make me worry is that I'm sure there are
>>> other sites at least as much "big" as the one Philippe
>>> speaks about (but it's
>>> true it's a big one, at least in OFBiz world).
>>>
>>> Others opinions ?
>>>
>>> Jacques
>>>
>>>
>>>
>>
>>
>>
>>
>
>
>

--
Pierre Gaudin,
Consultant Fonctionnel Neogia, Apache-OFBiz
ERP en logiciel Libre

Société Néréide
mobile : [phone]
bureau : [phone]
http://www.nereide.biz 

[Jacques Le Roux]

Hi Pierre,

Yes I know that. That's why I will commit soon... Tomorrow I guess, anyway it's there for months...

Thanks!

Jacques

From: "pierre.gaudin" <[hidden email]>

> Hi  Jacques,
>
> I know that it is not the proof that functions correctly but we work with Philippe on joint projects and the patchs have been in
> production on OFBiz for several months without any problems.
>
> Regards
>
> Pierre
> Jacques Le Roux a écrit :
>> Thanks for comment Adrian,
>>
>> I really think Philippe (and his team) has done a good work. I trust them and I'd like to put that in. I was looking for cons if
>> any, else I will commit soon...
>>
>> Thanks
>>
>> Jacques
>>
>> From: "Adrian Crum" <[hidden email]>
>>> There is no doubt that there are synchronization issues in the Java code. I run across them all the time and fix them as I find
>>> them.
>>>
>>> Maybe this is a good time to start a synchronization "Best Practices" thread where we can all offer our insights and suggestions
>>> so that poorly synchronized code is easier to notice and correct.
>>>
>>> -Adrian
>>>
>>>
>>> --- On Sat, 11/14/09, Jacques Le Roux <[hidden email]> wrote:
>>>
>>>> From: Jacques Le Roux <[hidden email]>
>>>> Subject: Re: Synchronisation issues
>>>> To: [hidden email]
>>>> Date: Saturday, November 14, 2009, 6:58 AM
>>>> From: "Adam Heath" <[hidden email]>
>>>> >> But before that I'd be interested to have some
>>>> opinions by people who
>>>> >> are using load balancers. Notably Brett who
>>>> recently answered about this
>>>> >> subject on user ML. Because I wonder why Philippe
>>>> was the 1st to
>>>> >> discover theses issues and if others have fixed
>>>> them in another way.
>>>> >
>>>> > Using synchronization can cause deadlocks. If
>>>> you do not *fully*
>>>> > control *all* code that is run under synchronization,
>>>> then a deadlock
>>>> > can occur.
>>>>
>>>> I agree, synchronization is tricky. BTW, what to you think
>>>> about r751619 ? Here is a remark about it
>>>> https://issues.apache.org/jira/browse/OFBIZ-2186?focusedCommentId=12680105&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12680105
>>>>
>>>> > If you call some other library, and said library is
>>>> extensible, such
>>>> > that it might end up calling *back* into your code,
>>>> you *will* deadlock.
>>>>
>>>> I agree, but I don't think it concerns this problem (I
>>>> checked the Call Hierarchy)
>>>>
>>>> > Non-blocking algorythms don't have this problem,
>>>> because they retry in
>>>> > these cases.
>>>>
>>>> Sure, but a problem appeared here, it's well explained at
>>>> https://issues.apache.org/jira/browse/OFBIZ-2124 and
>>>> they propose a fix
>>>> that works.
>>>> It's not the 1st time this kind of problem appears. We have
>>>> an opportunity to try a fix, this is the pragmatic reason of
>>>> this
>>>> message. What make me worry is that I'm sure there are
>>>> other sites at least as much "big" as the one Philippe
>>>> speaks about (but it's
>>>> true it's a big one, at least in OFBiz world).
>>>>
>>>> Others opinions ?
>>>>
>>>> Jacques
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>
>
> --
> Pierre Gaudin,
> Consultant Fonctionnel Neogia, Apache-OFBiz
> ERP en logiciel Libre
>
> Société Néréide
> mobile : [phone]
> bureau : [phone]
> http://www.nereide.biz
>

[pierre-29]

Thank you very much for all this work Jacques!

Pierre
Jacques Le Roux a écrit :

> Hi Pierre,
>
> Yes I know that. That's why I will commit soon... Tomorrow I guess,
> anyway it's there for months...
>
> Thanks!
>
> Jacques
>
> From: "pierre.gaudin" <[hidden email]>
>> Hi  Jacques,
>>
>> I know that it is not the proof that functions correctly but we work
>> with Philippe on joint projects and the patchs have been in
>> production on OFBiz for several months without any problems.
>>
>> Regards
>>
>> Pierre
>> Jacques Le Roux a écrit :
>>> Thanks for comment Adrian,
>>>
>>> I really think Philippe (and his team) has done a good work. I trust
>>> them and I'd like to put that in. I was looking for cons if
>>> any, else I will commit soon...
>>>
>>> Thanks
>>>
>>> Jacques
>>>
>>> From: "Adrian Crum" <[hidden email]>
>>>> There is no doubt that there are synchronization issues in the Java
>>>> code. I run across them all the time and fix them as I find
>>>> them.
>>>>
>>>> Maybe this is a good time to start a synchronization "Best
>>>> Practices" thread where we can all offer our insights and suggestions
>>>> so that poorly synchronized code is easier to notice and correct.
>>>>
>>>> -Adrian
>>>>
>>>>
>>>> --- On Sat, 11/14/09, Jacques Le Roux
>>>> <[hidden email]> wrote:
>>>>
>>>>> From: Jacques Le Roux <[hidden email]>
>>>>> Subject: Re: Synchronisation issues
>>>>> To: [hidden email]
>>>>> Date: Saturday, November 14, 2009, 6:58 AM
>>>>> From: "Adam Heath" <[hidden email]>
>>>>> >> But before that I'd be interested to have some
>>>>> opinions by people who
>>>>> >> are using load balancers. Notably Brett who
>>>>> recently answered about this
>>>>> >> subject on user ML. Because I wonder why Philippe
>>>>> was the 1st to
>>>>> >> discover theses issues and if others have fixed
>>>>> them in another way.
>>>>> >
>>>>> > Using synchronization can cause deadlocks. If
>>>>> you do not *fully*
>>>>> > control *all* code that is run under synchronization,
>>>>> then a deadlock
>>>>> > can occur.
>>>>>
>>>>> I agree, synchronization is tricky. BTW, what to you think
>>>>> about r751619 ? Here is a remark about it
>>>>> https://issues.apache.org/jira/browse/OFBIZ-2186?focusedCommentId=12680105&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12680105 
>>>>>
>>>>>
>>>>> > If you call some other library, and said library is
>>>>> extensible, such
>>>>> > that it might end up calling *back* into your code,
>>>>> you *will* deadlock.
>>>>>
>>>>> I agree, but I don't think it concerns this problem (I
>>>>> checked the Call Hierarchy)
>>>>>
>>>>> > Non-blocking algorythms don't have this problem,
>>>>> because they retry in
>>>>> > these cases.
>>>>>
>>>>> Sure, but a problem appeared here, it's well explained at
>>>>> https://issues.apache.org/jira/browse/OFBIZ-2124 and
>>>>> they propose a fix
>>>>> that works.
>>>>> It's not the 1st time this kind of problem appears. We have
>>>>> an opportunity to try a fix, this is the pragmatic reason of
>>>>> this
>>>>> message. What make me worry is that I'm sure there are
>>>>> other sites at least as much "big" as the one Philippe
>>>>> speaks about (but it's
>>>>> true it's a big one, at least in OFBiz world).
>>>>>
>>>>> Others opinions ?
>>>>>
>>>>> Jacques
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>
>>
>> --
>> Pierre Gaudin,
>> Consultant Fonctionnel Neogia, Apache-OFBiz
>> ERP en logiciel Libre
>>
>> Société Néréide
>> mobile : [phone]
>> bureau : [phone]
>> http://www.nereide.biz
>>
>
>
>

--
Pierre Gaudin,
Consultant Fonctionnel Neogia, Apache-OFBiz
ERP en logiciel Libre

Société Néréide
mobile : [phone]
bureau : [phone]
http://www.nereide.biz 

[Jacques Le Roux]

Hi Pierre,

Thanks for you encouragements.

All,

I haven't close yet https://issues.apache.org/jira/browse/OFBIZ-2124  because hopefully Philippe will submit a patch for R4.0 (I
know this project used R4, and I guess it still do)

There is still a related issue opened https://issues.apache.org/jira/browse/OFBIZ-2186
From Philippe informations I'd like to commit this patch also. I will do soon, except if someone see really something bad in it.
It's about possible synchronisation slowness, but as they checked it seriously (see last comment) I'm quite sure we can trust them.

Jacques

From: "pierre" <[hidden email]>

> Thank you very much for all this work Jacques!
>
> Pierre
> Jacques Le Roux a écrit :
>> Hi Pierre,
>>
>> Yes I know that. That's why I will commit soon... Tomorrow I guess, anyway it's there for months...
>>
>> Thanks!
>>
>> Jacques
>>
>> From: "pierre.gaudin" <[hidden email]>
>>> Hi  Jacques,
>>>
>>> I know that it is not the proof that functions correctly but we work with Philippe on joint projects and the patchs have been in
>>> production on OFBiz for several months without any problems.
>>>
>>> Regards
>>>
>>> Pierre
>>> Jacques Le Roux a écrit :
>>>> Thanks for comment Adrian,
>>>>
>>>> I really think Philippe (and his team) has done a good work. I trust them and I'd like to put that in. I was looking for cons
>>>> if
>>>> any, else I will commit soon...
>>>>
>>>> Thanks
>>>>
>>>> Jacques
>>>>
>>>> From: "Adrian Crum" <[hidden email]>
>>>>> There is no doubt that there are synchronization issues in the Java code. I run across them all the time and fix them as I
>>>>> find
>>>>> them.
>>>>>
>>>>> Maybe this is a good time to start a synchronization "Best Practices" thread where we can all offer our insights and
>>>>> suggestions
>>>>> so that poorly synchronized code is easier to notice and correct.
>>>>>
>>>>> -Adrian
>>>>>
>>>>>
>>>>> --- On Sat, 11/14/09, Jacques Le Roux <[hidden email]> wrote:
>>>>>
>>>>>> From: Jacques Le Roux <[hidden email]>
>>>>>> Subject: Re: Synchronisation issues
>>>>>> To: [hidden email]
>>>>>> Date: Saturday, November 14, 2009, 6:58 AM
>>>>>> From: "Adam Heath" <[hidden email]>
>>>>>> >> But before that I'd be interested to have some
>>>>>> opinions by people who
>>>>>> >> are using load balancers. Notably Brett who
>>>>>> recently answered about this
>>>>>> >> subject on user ML. Because I wonder why Philippe
>>>>>> was the 1st to
>>>>>> >> discover theses issues and if others have fixed
>>>>>> them in another way.
>>>>>> >
>>>>>> > Using synchronization can cause deadlocks. If
>>>>>> you do not *fully*
>>>>>> > control *all* code that is run under synchronization,
>>>>>> then a deadlock
>>>>>> > can occur.
>>>>>>
>>>>>> I agree, synchronization is tricky. BTW, what to you think
>>>>>> about r751619 ? Here is a remark about it
>>>>>> https://issues.apache.org/jira/browse/OFBIZ-2186?focusedCommentId=12680105&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12680105
>>>>>>
>>>>>> > If you call some other library, and said library is
>>>>>> extensible, such
>>>>>> > that it might end up calling *back* into your code,
>>>>>> you *will* deadlock.
>>>>>>
>>>>>> I agree, but I don't think it concerns this problem (I
>>>>>> checked the Call Hierarchy)
>>>>>>
>>>>>> > Non-blocking algorythms don't have this problem,
>>>>>> because they retry in
>>>>>> > these cases.
>>>>>>
>>>>>> Sure, but a problem appeared here, it's well explained at
>>>>>> https://issues.apache.org/jira/browse/OFBIZ-2124 and
>>>>>> they propose a fix
>>>>>> that works.
>>>>>> It's not the 1st time this kind of problem appears. We have
>>>>>> an opportunity to try a fix, this is the pragmatic reason of
>>>>>> this
>>>>>> message. What make me worry is that I'm sure there are
>>>>>> other sites at least as much "big" as the one Philippe
>>>>>> speaks about (but it's
>>>>>> true it's a big one, at least in OFBiz world).
>>>>>>
>>>>>> Others opinions ?
>>>>>>
>>>>>> Jacques
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Pierre Gaudin,
>>> Consultant Fonctionnel Neogia, Apache-OFBiz
>>> ERP en logiciel Libre
>>>
>>> Société Néréide
>>> mobile : [phone]
>>> bureau : [phone]
>>> http://www.nereide.biz
>>>
>>
>>
>>
>
>
> --
> Pierre Gaudin,
> Consultant Fonctionnel Neogia, Apache-OFBiz
> ERP en logiciel Libre
>
> Société Néréide
> mobile : [phone]
> bureau : [phone]
> http://www.nereide.biz
>