Performance issue: org.ofbiz.entity.transaction.TransactionUtil seems to be bottleneck

[Martin Kreidenweis]

Hi,

we are currently evaluating Apache OFBiz for use in one of our projects.

We expect a high load on our application. So we did a test with jMeter
on the sample ecommerce application with 100 parallel sessions and in
average 400 msec (randomized) wait time between requests.

During profiling we experienced that
org.ofbiz.entity.transaction.TransactionUtil.begin() is by far the
dominating method invoked and the application spends about 75% of total
time in this method.
This results in page load times over 10 seconds and growing.

Many threads are blocked because the method is defined as static and
synchronized.

We are using MySQL as a database backend in our test setup and
deactivated all informational logging (even server hit statistics), and
most of the sidebar widgets.

Any suggestions/ideas are highly appreciated.
Thanks.

Best regards,
Martin

[Jacques Le Roux]

1st level of answer, did you set caches?
https://cwiki.apache.org/confluence/display/OFBTECH/Apache+OFBiz+Technical+Production+Setup+Guide#ApacheOFBizTechnicalProductionSetupGuide-CacheSettings
https://cwiki.apache.org/confluence/display/OFBADMIN/Scaling+and+Performance+Plan

Jacques

From: "Martin Kreidenweis" <[hidden email]>

> Hi,
>
> we are currently evaluating Apache OFBiz for use in one of our projects.
>
> We expect a high load on our application. So we did a test with jMeter
> on the sample ecommerce application with 100 parallel sessions and in
> average 400 msec (randomized) wait time between requests.
>
> During profiling we experienced that
> org.ofbiz.entity.transaction.TransactionUtil.begin() is by far the
> dominating method invoked and the application spends about 75% of total
> time in this method.
> This results in page load times over 10 seconds and growing.
>
> Many threads are blocked because the method is defined as static and
> synchronized.
>
> We are using MySQL as a database backend in our test setup and
> deactivated all informational logging (even server hit statistics), and
> most of the sidebar widgets.
>
> Any suggestions/ideas are highly appreciated.
> Thanks.
>
> Best regards,
> Martin
>

[Karl Pitrich]

Hi,

I'm working with Martin on this project.

We have commented the various cache options as described in the Wiki (and mailing list),
EXPIRE TIME is 0 in all except for product.* when viewing /webtools/control/FindUtilCache

Logging is disabled, as are several components.
Also, we have disabled several content elements from the template, such as recommendations or polls, etc.


Any help is appreciated,
Greetings,

 - Karl

On 31.05.2010, at 22:07, Jacques Le Roux wrote:

> 1st level of answer, did you set caches?
> https://cwiki.apache.org/confluence/display/OFBTECH/Apache+OFBiz+Technical+Production+Setup+Guide#ApacheOFBizTechnicalProductionSetupGuide-CacheSettings
> https://cwiki.apache.org/confluence/display/OFBADMIN/Scaling+and+Performance+Plan
>
> Jacques
>
> From: "Martin Kreidenweis" <[hidden email]>
>> Hi,
>>
>> we are currently evaluating Apache OFBiz for use in one of our projects.
>>
>> We expect a high load on our application. So we did a test with jMeter
>> on the sample ecommerce application with 100 parallel sessions and in
>> average 400 msec (randomized) wait time between requests.
>>
>> During profiling we experienced that
>> org.ofbiz.entity.transaction.TransactionUtil.begin() is by far the
>> dominating method invoked and the application spends about 75% of total
>> time in this method.
>> This results in page load times over 10 seconds and growing.
>>
>> Many threads are blocked because the method is defined as static and
>> synchronized.
>>
>> We are using MySQL as a database backend in our test setup and
>> deactivated all informational logging (even server hit statistics), and
>> most of the sidebar widgets.
>>
>> Any suggestions/ideas are highly appreciated.
>> Thanks.
>>
>> Best regards,
>> Martin

[David E. Jones-2]

The theory that Martin mentioned is interesting, and quite possible. Have you tried removing the synchronized keyword from the begin method to see if it helps?

That would just be to test the theory, and if that does turn out to be the bottleneck then the sensitive parts of the method should be synchronized (or put into separate synchronized methods) instead of snycing the whole method.

-David


On May 31, 2010, at 2:24 PM, Karl Pitrich wrote:

> Hi,
>
> I'm working with Martin on this project.
>
> We have commented the various cache options as described in the Wiki (and mailing list),
> EXPIRE TIME is 0 in all except for product.* when viewing /webtools/control/FindUtilCache
>
> Logging is disabled, as are several components.
> Also, we have disabled several content elements from the template, such as recommendations or polls, etc.
>
>
> Any help is appreciated,
> Greetings,
>
> - Karl
>
> On 31.05.2010, at 22:07, Jacques Le Roux wrote:
>
>> 1st level of answer, did you set caches?
>> https://cwiki.apache.org/confluence/display/OFBTECH/Apache+OFBiz+Technical+Production+Setup+Guide#ApacheOFBizTechnicalProductionSetupGuide-CacheSettings
>> https://cwiki.apache.org/confluence/display/OFBADMIN/Scaling+and+Performance+Plan
>>
>> Jacques
>>
>> From: "Martin Kreidenweis" <[hidden email]>
>>> Hi,
>>>
>>> we are currently evaluating Apache OFBiz for use in one of our projects.
>>>
>>> We expect a high load on our application. So we did a test with jMeter
>>> on the sample ecommerce application with 100 parallel sessions and in
>>> average 400 msec (randomized) wait time between requests.
>>>
>>> During profiling we experienced that
>>> org.ofbiz.entity.transaction.TransactionUtil.begin() is by far the
>>> dominating method invoked and the application spends about 75% of total
>>> time in this method.
>>> This results in page load times over 10 seconds and growing.
>>>
>>> Many threads are blocked because the method is defined as static and
>>> synchronized.
>>>
>>> We are using MySQL as a database backend in our test setup and
>>> deactivated all informational logging (even server hit statistics), and
>>> most of the sidebar widgets.
>>>
>>> Any suggestions/ideas are highly appreciated.
>>> Thanks.
>>>
>>> Best regards,
>>> Martin

[rrhati2010]

Thanks for the findings and suggestions....even I am facing the same problem deploying in Jboss4.2 and tested with Jmeter for it's performance.

Everytime I start putting load on server it gets me Error: OutofMemory or OutofSwapSpace or Not able to create a new native thread even though there is enough heap memory(around 512M free).

I have raised this issue on JIRA also: OFBIZ-3707

Further I will try to implement your suggestions and let you know the results.

Thanks,
RRH

[Deyan Tsvetanov-3]

Hi guys,  

removing synchronized from begin( int timeout) will cause issues.
The problematic code is the following:
(starting from line 180 in TransactionUtil.java  - trunk )

 // reset the transaction stamps, just in case...
             
clearTransactionStamps();
// initialize the start stamp
getTransactionStartStamp();
// set the tx begin stack placeholder
setTransactionBeginStack();

So the synchronized begin() is a huge bottleneck and imho it can't be
solved without coding. Caching is not a solution.
I think the transaction timestamps and the transaction begin stack have
to be moved to the UserTransaction so they are not static anymore.
In general TransactionUtil needs some rework.

Cheers,
Deyan

On Mon, 2010-05-31 at 14:34 -0600, David E Jones wrote:

> The theory that Martin mentioned is interesting, and quite possible. Have you tried removing the synchronized keyword from the begin method to see if it helps?
>
> That would just be to test the theory, and if that does turn out to be the bottleneck then the sensitive parts of the method should be synchronized (or put into separate synchronized methods) instead of snycing the whole method.
>
> -David
>
>
> On May 31, 2010, at 2:24 PM, Karl Pitrich wrote:
>
> > Hi,
> >
> > I'm working with Martin on this project.
> >
> > We have commented the various cache options as described in the Wiki (and mailing list),
> > EXPIRE TIME is 0 in all except for product.* when viewing /webtools/control/FindUtilCache
> >
> > Logging is disabled, as are several components.
> > Also, we have disabled several content elements from the template, such as recommendations or polls, etc.
> >
> >
> > Any help is appreciated,
> > Greetings,
> >
> > - Karl
> >
> > On 31.05.2010, at 22:07, Jacques Le Roux wrote:
> >
> >> 1st level of answer, did you set caches?
> >> https://cwiki.apache.org/confluence/display/OFBTECH/Apache+OFBiz+Technical+Production+Setup+Guide#ApacheOFBizTechnicalProductionSetupGuide-CacheSettings
> >> https://cwiki.apache.org/confluence/display/OFBADMIN/Scaling+and+Performance+Plan
> >>
> >> Jacques
> >>
> >> From: "Martin Kreidenweis" <[hidden email]>
> >>> Hi,
> >>>
> >>> we are currently evaluating Apache OFBiz for use in one of our projects.
> >>>
> >>> We expect a high load on our application. So we did a test with jMeter
> >>> on the sample ecommerce application with 100 parallel sessions and in
> >>> average 400 msec (randomized) wait time between requests.
> >>>
> >>> During profiling we experienced that
> >>> org.ofbiz.entity.transaction.TransactionUtil.begin() is by far the
> >>> dominating method invoked and the application spends about 75% of total
> >>> time in this method.
> >>> This results in page load times over 10 seconds and growing.
> >>>
> >>> Many threads are blocked because the method is defined as static and
> >>> synchronized.
> >>>
> >>> We are using MySQL as a database backend in our test setup and
> >>> deactivated all informational logging (even server hit statistics), and
> >>> most of the sidebar widgets.
> >>>
> >>> Any suggestions/ideas are highly appreciated.
> >>> Thanks.
> >>>
> >>> Best regards,
> >>> Martin
>

[Deyan Tsvetanov-3]

I take my words back about the workaround.
It's much more complicated.

On Tue, 2010-06-01 at 09:23 +0300, Deyan Tsvetanov wrote:

> Hi guys,  
>
> removing synchronized from begin( int timeout) will cause issues.
> The problematic code is the following:
> (starting from line 180 in TransactionUtil.java  - trunk )
>
>  // reset the transaction stamps, just in case...
>              
> clearTransactionStamps();
> // initialize the start stamp
> getTransactionStartStamp();
> // set the tx begin stack placeholder
> setTransactionBeginStack();
>
> So the synchronized begin() is a huge bottleneck and imho it can't be
> solved without coding. Caching is not a solution.
> I think the transaction timestamps and the transaction begin stack have
> to be moved to the UserTransaction so they are not static anymore.
> In general TransactionUtil needs some rework.
>
> Cheers,
> Deyan
>
> On Mon, 2010-05-31 at 14:34 -0600, David E Jones wrote:
> > The theory that Martin mentioned is interesting, and quite possible. Have you tried removing the synchronized keyword from the begin method to see if it helps?
> >
> > That would just be to test the theory, and if that does turn out to be the bottleneck then the sensitive parts of the method should be synchronized (or put into separate synchronized methods) instead of snycing the whole method.
> >
> > -David
> >
> >
> > On May 31, 2010, at 2:24 PM, Karl Pitrich wrote:
> >
> > > Hi,
> > >
> > > I'm working with Martin on this project.
> > >
> > > We have commented the various cache options as described in the Wiki (and mailing list),
> > > EXPIRE TIME is 0 in all except for product.* when viewing /webtools/control/FindUtilCache
> > >
> > > Logging is disabled, as are several components.
> > > Also, we have disabled several content elements from the template, such as recommendations or polls, etc.
> > >
> > >
> > > Any help is appreciated,
> > > Greetings,
> > >
> > > - Karl
> > >
> > > On 31.05.2010, at 22:07, Jacques Le Roux wrote:
> > >
> > >> 1st level of answer, did you set caches?
> > >> https://cwiki.apache.org/confluence/display/OFBTECH/Apache+OFBiz+Technical+Production+Setup+Guide#ApacheOFBizTechnicalProductionSetupGuide-CacheSettings
> > >> https://cwiki.apache.org/confluence/display/OFBADMIN/Scaling+and+Performance+Plan
> > >>
> > >> Jacques
> > >>
> > >> From: "Martin Kreidenweis" <[hidden email]>
> > >>> Hi,
> > >>>
> > >>> we are currently evaluating Apache OFBiz for use in one of our projects.
> > >>>
> > >>> We expect a high load on our application. So we did a test with jMeter
> > >>> on the sample ecommerce application with 100 parallel sessions and in
> > >>> average 400 msec (randomized) wait time between requests.
> > >>>
> > >>> During profiling we experienced that
> > >>> org.ofbiz.entity.transaction.TransactionUtil.begin() is by far the
> > >>> dominating method invoked and the application spends about 75% of total
> > >>> time in this method.
> > >>> This results in page load times over 10 seconds and growing.
> > >>>
> > >>> Many threads are blocked because the method is defined as static and
> > >>> synchronized.
> > >>>
> > >>> We are using MySQL as a database backend in our test setup and
> > >>> deactivated all informational logging (even server hit statistics), and
> > >>> most of the sidebar widgets.
> > >>>
> > >>> Any suggestions/ideas are highly appreciated.
> > >>> Thanks.
> > >>>
> > >>> Best regards,
> > >>> Martin
> >
>
>

[Deyan Tsvetanov-3]

Hi rhh,  

this looks like a different issue.
It is a memory leak and imho has nothing to do with the synchronized
TransactionUtil.begin() and commit() methods.


First you have to check out the max heap memory set for JBoss. Look for
an option -Xmx256m.
Increase it to 1024m.

If the test still produces OutOfMemory than there is a memory leak for
sure.

-- Deyan


On Mon, 2010-05-31 at 22:35 -0700, rrhati2010 wrote:

> Thanks for the findings and suggestions....even I am facing the same problem
> deploying in Jboss4.2 and tested with Jmeter for it's performance.
>
> Everytime I start putting load on server it gets me Error: OutofMemory or
> OutofSwapSpace or Not able to create a new native thread even though there
> is enough heap memory(around 512M free).
>
> I have raised this issue on JIRA also: OFBIZ-3707
>
> Further I will try to implement your suggestions and let you know the
> results.
>
> Thanks,
> RRH
>
>
> -----
> RRH

[rrhati2010]

Hi  Deyan,

I have tried out these things i.e (option -Xmx256m Increase it to 1024m).

The Server runs fine with 2-5 users load, but goes out of memory for 100 users load when tested with Jmeter for it's performance.

Do I need to profile the Ofbiz code.

Or, Can u suggest any other solution.

[Deyan Tsvetanov-3]

Hi rhh,

yes, it looks like a memory leak.
When you have a memory leak you have to profile each component you have
doubts about.

What screens have you tested with jMeter ?
You have to run a profiler, start the jMeter test again and if possible
- share the results. After that we could analyze the code and see if
there is any memory leak.

-- Deyan

On Tue, 2010-06-01 at 00:05 -0700, rrhati2010 wrote:

> Hi  Deyan,
>
> I have tried out these things i.e (option -Xmx256m Increase it to 1024m).
>
> The Server runs fine with 2-5 users load, but goes out of memory for 100
> users load when tested with Jmeter for it's performance.
>
> Do I need to profile the Ofbiz code.
>
> Or, Can u suggest any other solution.
>
> -----
> RRH

[Martin Kreidenweis]

Hi,

we probably found a simple solution for the synchronized begin()
problem. We set the TransactionUtil.debugResources flag to false; so no
more DebugXaResources are created in the begin() method, which seems to
be the problematic (performance-wise) part.
Now this method is a lot less problematic, judging from the profiler output.
I'm not exactly sure what the code does that's not run any more. But
judging from its name ("debug") it shouldn't have any adverse effects in
a production setting, should it?

Best Regards
Martin


On 01.06.2010 08:23, Deyan Tsvetanov wrote:

> Hi guys,  
>
> removing synchronized from begin( int timeout) will cause issues.
> The problematic code is the following:
> (starting from line 180 in TransactionUtil.java  - trunk )
>
>  // reset the transaction stamps, just in case...
>              
> clearTransactionStamps();
> // initialize the start stamp
> getTransactionStartStamp();
> // set the tx begin stack placeholder
> setTransactionBeginStack();
>
> So the synchronized begin() is a huge bottleneck and imho it can't be
> solved without coding. Caching is not a solution.
> I think the transaction timestamps and the transaction begin stack have
> to be moved to the UserTransaction so they are not static anymore.
> In general TransactionUtil needs some rework.
>
> Cheers,
> Deyan
>
> On Mon, 2010-05-31 at 14:34 -0600, David E Jones wrote:
>> The theory that Martin mentioned is interesting, and quite possible. Have you tried removing the synchronized keyword from the begin method to see if it helps?
>>
>> That would just be to test the theory, and if that does turn out to be the bottleneck then the sensitive parts of the method should be synchronized (or put into separate synchronized methods) instead of snycing the whole method.
>>
>> -David
>>
>>
>> On May 31, 2010, at 2:24 PM, Karl Pitrich wrote:
>>
>>> Hi,
>>>
>>> I'm working with Martin on this project.
>>>
>>> We have commented the various cache options as described in the Wiki (and mailing list),
>>> EXPIRE TIME is 0 in all except for product.* when viewing /webtools/control/FindUtilCache
>>>
>>> Logging is disabled, as are several components.
>>> Also, we have disabled several content elements from the template, such as recommendations or polls, etc.
>>>
>>>
>>> Any help is appreciated,
>>> Greetings,
>>>
>>> - Karl
>>>
>>> On 31.05.2010, at 22:07, Jacques Le Roux wrote:
>>>
>>>> 1st level of answer, did you set caches?
>>>> https://cwiki.apache.org/confluence/display/OFBTECH/Apache+OFBiz+Technical+Production+Setup+Guide#ApacheOFBizTechnicalProductionSetupGuide-CacheSettings
>>>> https://cwiki.apache.org/confluence/display/OFBADMIN/Scaling+and+Performance+Plan
>>>>
>>>> Jacques
>>>>
>>>> From: "Martin Kreidenweis" <[hidden email]>
>>>>> Hi,
>>>>>
>>>>> we are currently evaluating Apache OFBiz for use in one of our projects.
>>>>>
>>>>> We expect a high load on our application. So we did a test with jMeter
>>>>> on the sample ecommerce application with 100 parallel sessions and in
>>>>> average 400 msec (randomized) wait time between requests.
>>>>>
>>>>> During profiling we experienced that
>>>>> org.ofbiz.entity.transaction.TransactionUtil.begin() is by far the
>>>>> dominating method invoked and the application spends about 75% of total
>>>>> time in this method.
>>>>> This results in page load times over 10 seconds and growing.
>>>>>
>>>>> Many threads are blocked because the method is defined as static and
>>>>> synchronized.
>>>>>
>>>>> We are using MySQL as a database backend in our test setup and
>>>>> deactivated all informational logging (even server hit statistics), and
>>>>> most of the sidebar widgets.
>>>>>
>>>>> Any suggestions/ideas are highly appreciated.
>>>>> Thanks.
>>>>>
>>>>> Best regards,
>>>>> Martin
>>

[David E. Jones-2]

To be clear, I wasn't recommending that we remove synchronized from the begin method in OFBiz, just that they remove it temporarily and then redo their performance tests to see if that really is the problem.

You're absolutely right that more work will be necessary to localize the synchronization to avoid bottlenecks on a larger block of code.

BTW, the three sub-method calls you mentioned should all only deal with thread-local variables... aren't those thread-safe to use?

-David


On Jun 1, 2010, at 12:42 AM, Deyan Tsvetanov wrote:

> I take my words back about the workaround.
> It's much more complicated.
>
> On Tue, 2010-06-01 at 09:23 +0300, Deyan Tsvetanov wrote:
>> Hi guys,  
>>
>> removing synchronized from begin( int timeout) will cause issues.
>> The problematic code is the following:
>> (starting from line 180 in TransactionUtil.java  - trunk )
>>
>> // reset the transaction stamps, just in case...
>>
>> clearTransactionStamps();
>> // initialize the start stamp
>> getTransactionStartStamp();
>> // set the tx begin stack placeholder
>> setTransactionBeginStack();
>>
>> So the synchronized begin() is a huge bottleneck and imho it can't be
>> solved without coding. Caching is not a solution.
>> I think the transaction timestamps and the transaction begin stack have
>> to be moved to the UserTransaction so they are not static anymore.
>> In general TransactionUtil needs some rework.
>>
>> Cheers,
>> Deyan
>>
>> On Mon, 2010-05-31 at 14:34 -0600, David E Jones wrote:
>>> The theory that Martin mentioned is interesting, and quite possible. Have you tried removing the synchronized keyword from the begin method to see if it helps?
>>>
>>> That would just be to test the theory, and if that does turn out to be the bottleneck then the sensitive parts of the method should be synchronized (or put into separate synchronized methods) instead of snycing the whole method.
>>>
>>> -David
>>>
>>>
>>> On May 31, 2010, at 2:24 PM, Karl Pitrich wrote:
>>>
>>>> Hi,
>>>>
>>>> I'm working with Martin on this project.
>>>>
>>>> We have commented the various cache options as described in the Wiki (and mailing list),
>>>> EXPIRE TIME is 0 in all except for product.* when viewing /webtools/control/FindUtilCache
>>>>
>>>> Logging is disabled, as are several components.
>>>> Also, we have disabled several content elements from the template, such as recommendations or polls, etc.
>>>>
>>>>
>>>> Any help is appreciated,
>>>> Greetings,
>>>>
>>>> - Karl
>>>>
>>>> On 31.05.2010, at 22:07, Jacques Le Roux wrote:
>>>>
>>>>> 1st level of answer, did you set caches?
>>>>> https://cwiki.apache.org/confluence/display/OFBTECH/Apache+OFBiz+Technical+Production+Setup+Guide#ApacheOFBizTechnicalProductionSetupGuide-CacheSettings
>>>>> https://cwiki.apache.org/confluence/display/OFBADMIN/Scaling+and+Performance+Plan
>>>>>
>>>>> Jacques
>>>>>
>>>>> From: "Martin Kreidenweis" <[hidden email]>
>>>>>> Hi,
>>>>>>
>>>>>> we are currently evaluating Apache OFBiz for use in one of our projects.
>>>>>>
>>>>>> We expect a high load on our application. So we did a test with jMeter
>>>>>> on the sample ecommerce application with 100 parallel sessions and in
>>>>>> average 400 msec (randomized) wait time between requests.
>>>>>>
>>>>>> During profiling we experienced that
>>>>>> org.ofbiz.entity.transaction.TransactionUtil.begin() is by far the
>>>>>> dominating method invoked and the application spends about 75% of total
>>>>>> time in this method.
>>>>>> This results in page load times over 10 seconds and growing.
>>>>>>
>>>>>> Many threads are blocked because the method is defined as static and
>>>>>> synchronized.
>>>>>>
>>>>>> We are using MySQL as a database backend in our test setup and
>>>>>> deactivated all informational logging (even server hit statistics), and
>>>>>> most of the sidebar widgets.
>>>>>>
>>>>>> Any suggestions/ideas are highly appreciated.
>>>>>> Thanks.
>>>>>>
>>>>>> Best regards,
>>>>>> Martin
>>>
>>
>>
>
>

[Deyan Tsvetanov-3]

Hi David,

yes, the thread-local vars are thread safe. However in order to assign
them a synchronized block or method is needed. There is no way to avoid
synchronized block if we want to create a transaction using a static
method without any arguments or without making the caller to keep a
return value in a local variable ( public void
TransactionUtil.begin() ). The begin() method needs to create a
thread-local var and associate it with the currently running thread.  

A workaround could be to use something like :

UserTransaction transaction = TransactionUtil.createNewTransaction();
//... execute code
TransactionUtil.commit( transaction );


this way we don't need any thread-local vars.
However this approach would require to make lots of fixes in the
existing code and this can not happen quickly.

We've got to think of something else.

-- Deyan

On Tue, 2010-06-01 at 03:16 -0600, David E Jones wrote:

> To be clear, I wasn't recommending that we remove synchronized from the begin method in OFBiz, just that they remove it temporarily and then redo their performance tests to see if that really is the problem.
>
> You're absolutely right that more work will be necessary to localize the synchronization to avoid bottlenecks on a larger block of code.
>
> BTW, the three sub-method calls you mentioned should all only deal with thread-local variables... aren't those thread-safe to use?
>
> -David
>
>
> On Jun 1, 2010, at 12:42 AM, Deyan Tsvetanov wrote:
>
> > I take my words back about the workaround.
> > It's much more complicated.
> >
> > On Tue, 2010-06-01 at 09:23 +0300, Deyan Tsvetanov wrote:
> >> Hi guys,  
> >>
> >> removing synchronized from begin( int timeout) will cause issues.
> >> The problematic code is the following:
> >> (starting from line 180 in TransactionUtil.java  - trunk )
> >>
> >> // reset the transaction stamps, just in case...
> >>
> >> clearTransactionStamps();
> >> // initialize the start stamp
> >> getTransactionStartStamp();
> >> // set the tx begin stack placeholder
> >> setTransactionBeginStack();
> >>
> >> So the synchronized begin() is a huge bottleneck and imho it can't be
> >> solved without coding. Caching is not a solution.
> >> I think the transaction timestamps and the transaction begin stack have
> >> to be moved to the UserTransaction so they are not static anymore.
> >> In general TransactionUtil needs some rework.
> >>
> >> Cheers,
> >> Deyan
> >>
> >> On Mon, 2010-05-31 at 14:34 -0600, David E Jones wrote:
> >>> The theory that Martin mentioned is interesting, and quite possible. Have you tried removing the synchronized keyword from the begin method to see if it helps?
> >>>
> >>> That would just be to test the theory, and if that does turn out to be the bottleneck then the sensitive parts of the method should be synchronized (or put into separate synchronized methods) instead of snycing the whole method.
> >>>
> >>> -David
> >>>
> >>>
> >>> On May 31, 2010, at 2:24 PM, Karl Pitrich wrote:
> >>>
> >>>> Hi,
> >>>>
> >>>> I'm working with Martin on this project.
> >>>>
> >>>> We have commented the various cache options as described in the Wiki (and mailing list),
> >>>> EXPIRE TIME is 0 in all except for product.* when viewing /webtools/control/FindUtilCache
> >>>>
> >>>> Logging is disabled, as are several components.
> >>>> Also, we have disabled several content elements from the template, such as recommendations or polls, etc.
> >>>>
> >>>>
> >>>> Any help is appreciated,
> >>>> Greetings,
> >>>>
> >>>> - Karl
> >>>>
> >>>> On 31.05.2010, at 22:07, Jacques Le Roux wrote:
> >>>>
> >>>>> 1st level of answer, did you set caches?
> >>>>> https://cwiki.apache.org/confluence/display/OFBTECH/Apache+OFBiz+Technical+Production+Setup+Guide#ApacheOFBizTechnicalProductionSetupGuide-CacheSettings
> >>>>> https://cwiki.apache.org/confluence/display/OFBADMIN/Scaling+and+Performance+Plan
> >>>>>
> >>>>> Jacques
> >>>>>
> >>>>> From: "Martin Kreidenweis" <[hidden email]>
> >>>>>> Hi,
> >>>>>>
> >>>>>> we are currently evaluating Apache OFBiz for use in one of our projects.
> >>>>>>
> >>>>>> We expect a high load on our application. So we did a test with jMeter
> >>>>>> on the sample ecommerce application with 100 parallel sessions and in
> >>>>>> average 400 msec (randomized) wait time between requests.
> >>>>>>
> >>>>>> During profiling we experienced that
> >>>>>> org.ofbiz.entity.transaction.TransactionUtil.begin() is by far the
> >>>>>> dominating method invoked and the application spends about 75% of total
> >>>>>> time in this method.
> >>>>>> This results in page load times over 10 seconds and growing.
> >>>>>>
> >>>>>> Many threads are blocked because the method is defined as static and
> >>>>>> synchronized.
> >>>>>>
> >>>>>> We are using MySQL as a database backend in our test setup and
> >>>>>> deactivated all informational logging (even server hit statistics), and
> >>>>>> most of the sidebar widgets.
> >>>>>>
> >>>>>> Any suggestions/ideas are highly appreciated.
> >>>>>> Thanks.
> >>>>>>
> >>>>>> Best regards,
> >>>>>> Martin
> >>>
> >>
> >>
> >
> >
>