OFBiz › OFBiz - User

Ofbiz performance test

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

9 messages Options

Vikrant.Rathore

Ofbiz performance test

Hi Everyone,

I have started working on creating a performance test for ofbiz along
with its development. I am using the tool http://www.webload.org/ to do
functional as well as performance testing. I could have done this in
Selenium or JMeter but it seems webload is polished, easy and now is
open source.

If anyone else has some performance figures for ofbiz please would you
share with me.

Thanks
with best regards,
Vikrant

David E Jones

Re: Ofbiz performance test

That's great Vikrant. Especially as we start working on stabilizing
the release branch testing in general will be very important.

Unfortunately WebLoad is GPL licensed. What that means is that not
only can we not include it with OFBiz, but we can't even include code
that uses it in OFBiz (because it would have to be GPL licensed, and
if other code used it that code would have to be GPL licensed... the
viral effect can kill our more friendly licensing).

-David

On Apr 18, 2007, at 7:39 PM, <[hidden email]>
<[hidden email]> wrote:

> Hi Everyone,
>
> I have started working on creating a performance test for ofbiz along
> with its development. I am using the tool http://www.webload.org/
> to do
> functional as well as performance testing. I could have done this in
> Selenium or JMeter but it seems webload is polished, easy and now is
> open source.
>
> If anyone else has some performance figures for ofbiz please would you
> share with me.
>
> Thanks
> with best regards,
> Vikrant
>

smime.p7s (3K) Download Attachment

Vikrant.Rathore

RE: Ofbiz performance test

In reply to this post by Vikrant.Rathore

Hi David,

Even if we do not include webload with ofbiz its not a problem, since we
can just include the test suites created for it.

I wanted to use JMeter (which can be obviously bundled with ofbiz), but
its not as polished as webload. Also webload is thick client application
and would require it to be installed on load machines, so it would be
good if we just include the test suite and let the users download it
themselves and test the application in their environment. Since at last
its upto the users to do the test based on their specific functional
requirements. Obviously we can use Webload to publish out performance
test results based on specific hardware and software configuration.

So GPL licensing won't pose any problem for us as we are not changing
the source of webload just using it as it is. Moreover we are not
bundling it with ofbiz (there is no need its better if user download by
themselves as it is pretty big application).

Thanks
with best regards,
Vikrant

-----Original Message-----
From: David E. Jones [mailto:[hidden email]]
Sent: Thursday, April 19, 2007 10:06 AM
To: [hidden email]
Subject: Re: Ofbiz performance test

That's great Vikrant. Especially as we start working on stabilizing the
release branch testing in general will be very important.

Unfortunately WebLoad is GPL licensed. What that means is that not only
can we not include it with OFBiz, but we can't even include code that
uses it in OFBiz (because it would have to be GPL licensed, and if other
code used it that code would have to be GPL licensed... the viral effect
can kill our more friendly licensing).

-David

On Apr 18, 2007, at 7:39 PM, <[hidden email]>
<[hidden email]> wrote:

> Hi Everyone,
>
> I have started working on creating a performance test for ofbiz along
> with its development. I am using the tool http://www.webload.org/ to
> do functional as well as performance testing. I could have done this
> in Selenium or JMeter but it seems webload is polished, easy and now
> is open source.
>
> If anyone else has some performance figures for ofbiz please would you

> share with me.
>
> Thanks
> with best regards,
> Vikrant
>

David E Jones

Re: Ofbiz performance test

One of the big goals for testing tools and a test suite in OFBiz is
to have tests that can be automatically run periodically by a server,
and preferably something that only requires a checkout, build, run
tests.

At this point we haven't a test tool that will fulfill this and other
things we'd like in an Apache compatible open source license.

As for WebLoad, it would depend on the licensing requirements of the
scripts it generates as sometimes those or parts of them are licensed
similar to what generates them.

-David

On Apr 18, 2007, at 8:31 PM, <[hidden email]>
<[hidden email]> wrote:

> Hi David,
>
> Even if we do not include webload with ofbiz its not a problem,
> since we
> can just include the test suites created for it.
>
> I wanted to use JMeter (which can be obviously bundled with ofbiz),
> but
> its not as polished as webload. Also webload is thick client
> application
> and would require it to be installed on load machines, so it would be
> good if we just include the test suite and let the users download it
> themselves and test the application in their environment. Since at
> last
> its upto the users to do the test based on their specific functional
> requirements. Obviously we can use Webload to publish out performance
> test results based on specific hardware and software configuration.
>
> So GPL licensing won't pose any problem for us as we are not changing
> the source of webload just using it as it is. Moreover we are not
> bundling it with ofbiz (there is no need its better if user
> download by
> themselves as it is pretty big application).
>
> Thanks
> with best regards,
> Vikrant
>
>
> -----Original Message-----
> From: David E. Jones [mailto:[hidden email]]
> Sent: Thursday, April 19, 2007 10:06 AM
> To: [hidden email]
> Subject: Re: Ofbiz performance test
>
>
> That's great Vikrant. Especially as we start working on stabilizing
> the
> release branch testing in general will be very important.
>
> Unfortunately WebLoad is GPL licensed. What that means is that not
> only
> can we not include it with OFBiz, but we can't even include code that
> uses it in OFBiz (because it would have to be GPL licensed, and if
> other
> code used it that code would have to be GPL licensed... the viral
> effect
> can kill our more friendly licensing).
>
> -David
>
>
> On Apr 18, 2007, at 7:39 PM, <[hidden email]>
> <[hidden email]> wrote:
>
>> Hi Everyone,
>>
>> I have started working on creating a performance test for ofbiz along
>> with its development. I am using the tool http://www.webload.org/ to
>> do functional as well as performance testing. I could have done this
>> in Selenium or JMeter but it seems webload is polished, easy and now
>> is open source.
>>
>> If anyone else has some performance figures for ofbiz please would
>> you
>
>> share with me.
>>
>> Thanks
>> with best regards,
>> Vikrant
>>
>

smime.p7s (3K) Download Attachment

Vikrant.Rathore

RE: Ofbiz performance test

In reply to this post by Vikrant.Rathore

Hi David,

Yes its a very good idea to periodically do a checkout build and run
tests. I think using JMeter can resolve this problem and I guess its
apache compatible license.

Currently my checkout, merge (with my own configuration changes), build
is done automatically. I think JMeter can be set to automatically run a
scheduled test and publish the results in html. Would check little
possibility and let you know.

Thanks
with best regards,
Vikrant

-----Original Message-----
From: David E. Jones [mailto:[hidden email]]
Sent: Thursday, April 19, 2007 4:20 PM
To: [hidden email]
Subject: Re: Ofbiz performance test

One of the big goals for testing tools and a test suite in OFBiz is to
have tests that can be automatically run periodically by a server, and
preferably something that only requires a checkout, build, run tests.

At this point we haven't a test tool that will fulfill this and other
things we'd like in an Apache compatible open source license.

As for WebLoad, it would depend on the licensing requirements of the
scripts it generates as sometimes those or parts of them are licensed
similar to what generates them.

-David

On Apr 18, 2007, at 8:31 PM, <[hidden email]>
<[hidden email]> wrote:

> Hi David,
>
> Even if we do not include webload with ofbiz its not a problem, since
> we can just include the test suites created for it.
>
> I wanted to use JMeter (which can be obviously bundled with ofbiz),
> but its not as polished as webload. Also webload is thick client
> application and would require it to be installed on load machines, so
> it would be good if we just include the test suite and let the users
> download it themselves and test the application in their environment.
> Since at last its upto the users to do the test based on their
> specific functional requirements. Obviously we can use Webload to
> publish out performance test results based on specific hardware and
> software configuration.
>
> So GPL licensing won't pose any problem for us as we are not changing
> the source of webload just using it as it is. Moreover we are not
> bundling it with ofbiz (there is no need its better if user download
> by themselves as it is pretty big application).
>
> Thanks
> with best regards,
> Vikrant
>
>
> -----Original Message-----
> From: David E. Jones [mailto:[hidden email]]
> Sent: Thursday, April 19, 2007 10:06 AM
> To: [hidden email]
> Subject: Re: Ofbiz performance test
>
>
> That's great Vikrant. Especially as we start working on stabilizing
> the release branch testing in general will be very important.
>
> Unfortunately WebLoad is GPL licensed. What that means is that not
> only can we not include it with OFBiz, but we can't even include code
> that uses it in OFBiz (because it would have to be GPL licensed, and
> if other code used it that code would have to be GPL licensed... the
> viral effect can kill our more friendly licensing).
>
> -David
>
>
> On Apr 18, 2007, at 7:39 PM, <[hidden email]>
> <[hidden email]> wrote:
>
>> Hi Everyone,
>>
>> I have started working on creating a performance test for ofbiz along

>> with its development. I am using the tool http://www.webload.org/ to
>> do functional as well as performance testing. I could have done this
>> in Selenium or JMeter but it seems webload is polished, easy and now
>> is open source.
>>
>> If anyone else has some performance figures for ofbiz please would
>> you
>
>> share with me.
>>
>> Thanks
>> with best regards,
>> Vikrant
>>
>

Vikrant.Rathore

RE: Ofbiz performance test

In reply to this post by Vikrant.Rathore

Hi David,

After checking out JMeter, it seems it can be run as ant task. The link
below is the Ant Task for JMeter. Also JMeter is apache TLP so bundling
it with ofbiz shoult not be an issue.

http://www.programmerplanet.org/pages/projects/jmeter-ant-task.php

http://jakarta.apache.org/jmeter/

Also we can built functional test using JMeter.

Thanks
with best regards,
Vikrant

-----Original Message-----
From: David E. Jones [mailto:[hidden email]]
Sent: Thursday, April 19, 2007 4:20 PM
To: [hidden email]
Subject: Re: Ofbiz performance test

One of the big goals for testing tools and a test suite in OFBiz is to
have tests that can be automatically run periodically by a server, and
preferably something that only requires a checkout, build, run tests.

At this point we haven't a test tool that will fulfill this and other
things we'd like in an Apache compatible open source license.

As for WebLoad, it would depend on the licensing requirements of the
scripts it generates as sometimes those or parts of them are licensed
similar to what generates them.

-David

On Apr 18, 2007, at 8:31 PM, <[hidden email]>
<[hidden email]> wrote:

G.Venkata Phanindra

Re: Ofbiz performance test

Hi,
Its very good idea to integrate JMeter with ofbiz .. when ever i had to use
JMeter for testing i use to run out of resources as i have to run them into
two different JVMs hopefully adding JMEter into ofbiz will solve the issue,
but i also have a question at the same time will it not force us to increase
the Memory size and also the resources while running ofbiz.

Regards
Phani

On 4/19/07, [hidden email] <
[hidden email]> wrote:

>
> Hi David,
>
> After checking out JMeter, it seems it can be run as ant task. The link
> below is the Ant Task for JMeter. Also JMeter is apache TLP so bundling
> it with ofbiz shoult not be an issue.
>
> http://www.programmerplanet.org/pages/projects/jmeter-ant-task.php
>
> http://jakarta.apache.org/jmeter/
>
> Also we can built functional test using JMeter.
>
> Thanks
> with best regards,
> Vikrant
>
>
> -----Original Message-----
> From: David E. Jones [mailto:[hidden email]]
> Sent: Thursday, April 19, 2007 4:20 PM
> To: [hidden email]
> Subject: Re: Ofbiz performance test
>
>
> One of the big goals for testing tools and a test suite in OFBiz is to
> have tests that can be automatically run periodically by a server, and
> preferably something that only requires a checkout, build, run tests.
>
> At this point we haven't a test tool that will fulfill this and other
> things we'd like in an Apache compatible open source license.
>
> As for WebLoad, it would depend on the licensing requirements of the
> scripts it generates as sometimes those or parts of them are licensed
> similar to what generates them.
>
> -David
>
>
> On Apr 18, 2007, at 8:31 PM, <[hidden email]>
> <[hidden email]> wrote:
>
> > Hi David,
> >
> > Even if we do not include webload with ofbiz its not a problem, since
> > we can just include the test suites created for it.
> >
> > I wanted to use JMeter (which can be obviously bundled with ofbiz),
> > but its not as polished as webload. Also webload is thick client
> > application and would require it to be installed on load machines, so
> > it would be good if we just include the test suite and let the users
> > download it themselves and test the application in their environment.
> > Since at last its upto the users to do the test based on their
> > specific functional requirements. Obviously we can use Webload to
> > publish out performance test results based on specific hardware and
> > software configuration.
> >
> > So GPL licensing won't pose any problem for us as we are not changing
> > the source of webload just using it as it is. Moreover we are not
> > bundling it with ofbiz (there is no need its better if user download
> > by themselves as it is pretty big application).
> >
> > Thanks
> > with best regards,
> > Vikrant
> >
> >
> > -----Original Message-----
> > From: David E. Jones [mailto:[hidden email]]
> > Sent: Thursday, April 19, 2007 10:06 AM
> > To: [hidden email]
> > Subject: Re: Ofbiz performance test
> >
> >
> > That's great Vikrant. Especially as we start working on stabilizing
> > the release branch testing in general will be very important.
> >
> > Unfortunately WebLoad is GPL licensed. What that means is that not
> > only can we not include it with OFBiz, but we can't even include code
> > that uses it in OFBiz (because it would have to be GPL licensed, and
> > if other code used it that code would have to be GPL licensed... the
> > viral effect can kill our more friendly licensing).
> >
> > -David
> >
> >
> > On Apr 18, 2007, at 7:39 PM, <[hidden email]>
> > <[hidden email]> wrote:
> >
> >> Hi Everyone,
> >>
> >> I have started working on creating a performance test for ofbiz along
>
> >> with its development. I am using the tool http://www.webload.org/ to
> >> do functional as well as performance testing. I could have done this
> >> in Selenium or JMeter but it seems webload is polished, easy and now
> >> is open source.
> >>
> >> If anyone else has some performance figures for ofbiz please would
> >> you
> >
> >> share with me.
> >>
> >> Thanks
> >> with best regards,
> >> Vikrant
> >>
> >
>
>

--
G.Venkata Phanindra
Mob:: 9849852989

Cameron Smith-6

Re: Ofbiz performance test

In reply to this post by Vikrant.Rathore

Dear Vikrant, the following information may or may not
be of use to you. We have one OFBiz-based product
which runs as a windows service on a laptop. If the
computer has at least 756Mb RAM, the application runs
satisfactorily once it gets going.

However the specific problem we had was startup time.
If the service took too long to drag itself up, the
user would click the shortcut in IE, and not get a
response from the Catalina container.

We managed to get the startup time down from about 45s
to 12s, with the measures described below. These
measurements were taken on a Fujitsu Siemens Amilo Pro
v2030, with a 1.7Ghz Mobile Celeron, and 1Gb RAM.
Windows XP, SP 2. It had several other applications
open at the same time, to simulate normal usage. We
used JDK 1.5.005 and MySQL 5.0.24.

Note that, if startup time is NOT your priority, some
of these measures may not be relevant.

1. Logging.
In debug.properties, log only to file and not stdout.
Set logging to WARN for everything.

2. Modules.
a. Remove all modules from hot-deploy and specialized
except the ones you actually use. You might also be
able to remove some of the modules in applications but
they do not have a hierarchical dependency, so you can
probably only remove one or two. We only removed the
marketing module, David has been quite clear about
this in previous posts - its generally more trouble
than its worth.
b. WITHIN the modules, deactivate any webapp you ain't
using (simply comment out the relevant part in
ofbiz-component.xml). For example, the only module we
left up, apart from our own custom module, was
webtools.

3. Containers
a. For a standard webapp system, we found that we can
get by with only the following 5 containers, we
removed everything else:
component-container
classloader-container
naming-container
catalina-container

b. Inside catalina-container, only activate the
containers you need - very often it is only http,
https or ajp, depending on your setup

4. Memory.
a. Give the JVM at least 384Mb of memory, otherwise it
really has trouble setting everything up. You can set
this either in startofbiz.bat/sh, or in service.conf
if you are running as a windows service (see my recent
tutorial on wiki).

There are some other points to explore, which for
various reasons I haven't tried yet.

5. Entity Engine
OFBiz does a lot of clever stuff when it starts up to
match XML entity definitions with what is actually in
the DB. But in a production system you aren't going
to change this very regularly. So theoretically you
should switch it off, and shave off a couple more
seconds at start up.
In practice, I only managed the following config for
my delegator in entityengine.xml:
check-on-start="true"
add-missing-on-start="true"
check-pks-on-start="false"

I wanted to set all 3 to false, but when I did this,
the JVM went into an endless loop and then bust with
an OutOfMemoryError. If anyone could shed any light
I'd be grateful.

6. JDK 1.6
I have run my application without any probs on JDK1.6,
which is supposed to up to a zillion % faster than
1.5 in certain situations according to SUN. But I
haven't taken any measurements yet, will let you know
when I do.

7. JNDI
From what I have seen of OFBiz internals so far, it
doesn't really use JNDI for much. So I will
investigate taking out this container as well, as for
my scenario, I am not getting anything from JNDI.

cameron

___________________________________________________________
Yahoo! Mail is the world's favourite email. Don't settle for less, sign up for
your free account today http://uk.rd.yahoo.com/evt=44106/*http://uk.docs.yahoo.com/mail/winter07.html

Vikrant.Rathore

RE: Ofbiz performance test

In reply to this post by Vikrant.Rathore

Hi Cameron,

Thanks very much for sharing the information. It will be very useful
information for us when we are implementing POS component for ofbiz as
startup time would matter a lot there.

On the server side we are not that much worried about the startup time
but more about the application performance when database grow
exponentially. To give you an idea we expect our database size would
reach atleast 600 GB in 1 year and also the number of active members
would be in upwards of 2.5 million. We do a lot of warehouse and
financial operations. So performance measurement is a must for us. Our
current implementation are all very slow indeed due to the transactions
and size of the database.

Thanks
with best regards,
Vikrant

-----Original Message-----
From: Cameron Smith [mailto:[hidden email]]
Sent: Thursday, April 19, 2007 6:34 PM
To: [hidden email]
Subject: Re: Ofbiz performance test

Dear Vikrant, the following information may or may not
be of use to you. We have one OFBiz-based product
which runs as a windows service on a laptop. If the
computer has at least 756Mb RAM, the application runs satisfactorily
once it gets going.

However the specific problem we had was startup time.
If the service took too long to drag itself up, the user would click the
shortcut in IE, and not get a response from the Catalina container.

We managed to get the startup time down from about 45s to 12s, with the
measures described below. These measurements were taken on a Fujitsu
Siemens Amilo Pro v2030, with a 1.7Ghz Mobile Celeron, and 1Gb RAM.
Windows XP, SP 2. It had several other applications open at the same
time, to simulate normal usage. We used JDK 1.5.005 and MySQL 5.0.24.

Note that, if startup time is NOT your priority, some of these measures
may not be relevant.

1. Logging.
In debug.properties, log only to file and not stdout.
Set logging to WARN for everything.

2. Modules.
a. Remove all modules from hot-deploy and specialized except the ones
you actually use. You might also be able to remove some of the modules
in applications but they do not have a hierarchical dependency, so you
can probably only remove one or two. We only removed the marketing
module, David has been quite clear about this in previous posts - its
generally more trouble than its worth.
b. WITHIN the modules, deactivate any webapp you ain't using (simply
comment out the relevant part in ofbiz-component.xml). For example, the
only module we left up, apart from our own custom module, was webtools.

3. Containers
a. For a standard webapp system, we found that we can get by with only
the following 5 containers, we removed everything else:
component-container
classloader-container
naming-container
catalina-container

b. Inside catalina-container, only activate the containers you need -
very often it is only http, https or ajp, depending on your setup

4. Memory.
a. Give the JVM at least 384Mb of memory, otherwise it really has
trouble setting everything up. You can set this either in
startofbiz.bat/sh, or in service.conf if you are running as a windows
service (see my recent tutorial on wiki).

There are some other points to explore, which for various reasons I
haven't tried yet.

5. Entity Engine
OFBiz does a lot of clever stuff when it starts up to match XML entity
definitions with what is actually in the DB. But in a production system
you aren't going to change this very regularly. So theoretically you
should switch it off, and shave off a couple more seconds at start up.
In practice, I only managed the following config for my delegator in
entityengine.xml:
check-on-start="true"
add-missing-on-start="true"
check-pks-on-start="false"

I wanted to set all 3 to false, but when I did this, the JVM went into
an endless loop and then bust with an OutOfMemoryError. If anyone could
shed any light I'd be grateful.

6. JDK 1.6
I have run my application without any probs on JDK1.6, which is
supposed to up to a zillion % faster than
1.5 in certain situations according to SUN. But I haven't taken any
measurements yet, will let you know when I do.

7. JNDI
From what I have seen of OFBiz internals so far, it doesn't really use
JNDI for much. So I will investigate taking out this container as well,
as for my scenario, I am not getting anything from JNDI.

cameron

___________________________________________________________
Yahoo! Mail is the world's favourite email. Don't settle for less, sign
up for your free account today
http://uk.rd.yahoo.com/evt=44106/*http://uk.docs.yahoo.com/mail/winter07
.html