Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh tools/git-rebase-runner.sh

[Adam Heath-2]

On 04/18/2012 02:57 AM, [hidden email] wrote:

> Author: jacopoc
> Date: Wed Apr 18 07:57:46 2012
> New Revision: 1327411
>
> URL: http://svn.apache.org/viewvc?rev=1327411&view=rev
> Log:
> Removed bin folder and moved its content to the tools folder: this was discussed extensively in the dev list recently.
>
> Added:
>     ofbiz/trunk/tools/functions.sh
>       - copied unchanged from r1327403, ofbiz/trunk/bin/functions.sh
>     ofbiz/trunk/tools/git-rebase-runner.sh
>       - copied unchanged from r1327403, ofbiz/trunk/bin/git-rebase-runner.sh
> Removed:
>     ofbiz/trunk/bin/

Sorry, missed that.  I've been away from ofbiz too long.

[Erwan de FERRIERES]

Le 18/04/2012 17:21, Adam Heath a écrit :

> On 04/18/2012 02:57 AM, [hidden email] wrote:
>> Author: jacopoc
>> Date: Wed Apr 18 07:57:46 2012
>> New Revision: 1327411
>>
>> URL: http://svn.apache.org/viewvc?rev=1327411&view=rev
>> Log:
>> Removed bin folder and moved its content to the tools folder: this was discussed extensively in the dev list recently.
>>
>> Added:
>>      ofbiz/trunk/tools/functions.sh
>>        - copied unchanged from r1327403, ofbiz/trunk/bin/functions.sh
>>      ofbiz/trunk/tools/git-rebase-runner.sh
>>        - copied unchanged from r1327403, ofbiz/trunk/bin/git-rebase-runner.sh
>> Removed:
>>      ofbiz/trunk/bin/
>
> Sorry, missed that.  I've been away from ofbiz too long.
>
>

So welcome back, then !

--
Erwan de FERRIERES
www.nereide.biz

[Adam Heath-2]

On 04/18/2012 10:55 AM, Erwan de FERRIERES wrote:

> Le 18/04/2012 17:21, Adam Heath a écrit :
>> On 04/18/2012 02:57 AM, [hidden email] wrote:
>>> Author: jacopoc
>>> Date: Wed Apr 18 07:57:46 2012
>>> New Revision: 1327411
>>>
>>> URL: http://svn.apache.org/viewvc?rev=1327411&view=rev
>>> Log:
>>> Removed bin folder and moved its content to the tools folder: this
>>> was discussed extensively in the dev list recently.
>>>
>>> Added:
>>>      ofbiz/trunk/tools/functions.sh
>>>        - copied unchanged from r1327403, ofbiz/trunk/bin/functions.sh
>>>      ofbiz/trunk/tools/git-rebase-runner.sh
>>>        - copied unchanged from r1327403,
>>> ofbiz/trunk/bin/git-rebase-runner.sh
>>> Removed:
>>>      ofbiz/trunk/bin/
>>
>> Sorry, missed that.  I've been away from ofbiz too long.
>>
>>
> So welcome back, then !

The main reason is the ofbiz community is not paying attention to how
slow the software has gotten.  It currently takes my desktop 1 hour to
run a full test suite.  clean, load-demo(30m), run-tests(30m).  It
used to take 12-15m.  This has caused me to not do much, as it takes
to long to do anything.

The issue that helps the most is getting a larger l2-cache on the cpu.
 The load-demo time is greatly affected by derby being *way* to large,
code-size wise.

I've started doing work now, by using a virtualized instance that has
more l2-cache, but the problem still remains.

[Jacopo Cappellato-4]

On Apr 18, 2012, at 6:43 PM, Adam Heath wrote:

> It currently takes my desktop 1 hour to
> run a full test suite.  clean, load-demo(30m), run-tests(30m).  It
> used to take 12-15m.

wow, it takes me 2-3 minutes in my laptop.

Jacopo

[Adrian Crum-3]

On 4/18/2012 5:43 PM, Adam Heath wrote:

> On 04/18/2012 10:55 AM, Erwan de FERRIERES wrote:
>> Le 18/04/2012 17:21, Adam Heath a écrit :
>>> On 04/18/2012 02:57 AM, [hidden email] wrote:
>>>> Author: jacopoc
>>>> Date: Wed Apr 18 07:57:46 2012
>>>> New Revision: 1327411
>>>>
>>>> URL: http://svn.apache.org/viewvc?rev=1327411&view=rev
>>>> Log:
>>>> Removed bin folder and moved its content to the tools folder: this
>>>> was discussed extensively in the dev list recently.
>>>>
>>>> Added:
>>>>       ofbiz/trunk/tools/functions.sh
>>>>         - copied unchanged from r1327403, ofbiz/trunk/bin/functions.sh
>>>>       ofbiz/trunk/tools/git-rebase-runner.sh
>>>>         - copied unchanged from r1327403,
>>>> ofbiz/trunk/bin/git-rebase-runner.sh
>>>> Removed:
>>>>       ofbiz/trunk/bin/
>>> Sorry, missed that.  I've been away from ofbiz too long.
>>>
>>>
>> So welcome back, then !
> The main reason is the ofbiz community is not paying attention to how
> slow the software has gotten.  It currently takes my desktop 1 hour to
> run a full test suite.  clean, load-demo(30m), run-tests(30m).  It
> used to take 12-15m.  This has caused me to not do much, as it takes
> to long to do anything.

Hard disk speed is what helps. With my high-speed SATA RAID array I can
build a fresh checkout and load demo data in less than 5 minutes. Ant
clean-data plus ant load-demo takes 4 minutes. The tests take 25 minutes
- but that is mainly due to the many timing loops involved. A faster
computer will not solve that.

-Adrian

[Jacques Le Roux]

Just buy a pair of SSDs, I can recommend Crucial M4 128Gb.
My old box (Intel Core Duo E8300 2.9 Mz from 2008) with XP and 4Go RAM is still happy, and it seems for long now... :o)
The disk subsystem has always been the bootleneck, not only on Windows...

The complete test takes me 32 minutes. I think with a faster/newer CPU could be twice less. I don't care, changing the machines does
not worth it (all that to move again, pfew I get too old, thanks SSDs...)

Jacques

From: "Adrian Crum" <[hidden email]>

> On 4/18/2012 5:43 PM, Adam Heath wrote:
>> On 04/18/2012 10:55 AM, Erwan de FERRIERES wrote:
>>> Le 18/04/2012 17:21, Adam Heath a écrit :
>>>> On 04/18/2012 02:57 AM, [hidden email] wrote:
>>>>> Author: jacopoc
>>>>> Date: Wed Apr 18 07:57:46 2012
>>>>> New Revision: 1327411
>>>>>
>>>>> URL: http://svn.apache.org/viewvc?rev=1327411&view=rev
>>>>> Log:
>>>>> Removed bin folder and moved its content to the tools folder: this
>>>>> was discussed extensively in the dev list recently.
>>>>>
>>>>> Added:
>>>>>       ofbiz/trunk/tools/functions.sh
>>>>>         - copied unchanged from r1327403, ofbiz/trunk/bin/functions.sh
>>>>>       ofbiz/trunk/tools/git-rebase-runner.sh
>>>>>         - copied unchanged from r1327403,
>>>>> ofbiz/trunk/bin/git-rebase-runner.sh
>>>>> Removed:
>>>>>       ofbiz/trunk/bin/
>>>> Sorry, missed that.  I've been away from ofbiz too long.
>>>>
>>>>
>>> So welcome back, then !
>> The main reason is the ofbiz community is not paying attention to how
>> slow the software has gotten.  It currently takes my desktop 1 hour to
>> run a full test suite.  clean, load-demo(30m), run-tests(30m).  It
>> used to take 12-15m.  This has caused me to not do much, as it takes
>> to long to do anything.
>
> Hard disk speed is what helps. With my high-speed SATA RAID array I can build a fresh checkout and load demo data in less than 5
> minutes. Ant clean-data plus ant load-demo takes 4 minutes. The tests take 25 minutes - but that is mainly due to the many timing
> loops involved. A faster computer will not solve that.
>
> -Adrian
>
>

[Adam Heath-2]

On 04/18/2012 11:56 AM, Adrian Crum wrote:

> On 4/18/2012 5:43 PM, Adam Heath wrote:
>> On 04/18/2012 10:55 AM, Erwan de FERRIERES wrote:
>>> Le 18/04/2012 17:21, Adam Heath a écrit :
>>>> On 04/18/2012 02:57 AM, [hidden email] wrote:
>>>>> Author: jacopoc
>>>>> Date: Wed Apr 18 07:57:46 2012
>>>>> New Revision: 1327411
>>>>>
>>>>> URL: http://svn.apache.org/viewvc?rev=1327411&view=rev
>>>>> Log:
>>>>> Removed bin folder and moved its content to the tools folder: this
>>>>> was discussed extensively in the dev list recently.
>>>>>
>>>>> Added:
>>>>>       ofbiz/trunk/tools/functions.sh
>>>>>         - copied unchanged from r1327403,
>>>>> ofbiz/trunk/bin/functions.sh
>>>>>       ofbiz/trunk/tools/git-rebase-runner.sh
>>>>>         - copied unchanged from r1327403,
>>>>> ofbiz/trunk/bin/git-rebase-runner.sh
>>>>> Removed:
>>>>>       ofbiz/trunk/bin/
>>>> Sorry, missed that.  I've been away from ofbiz too long.
>>>>
>>>>
>>> So welcome back, then !
>> The main reason is the ofbiz community is not paying attention to how
>> slow the software has gotten.  It currently takes my desktop 1 hour to
>> run a full test suite.  clean, load-demo(30m), run-tests(30m).  It
>> used to take 12-15m.  This has caused me to not do much, as it takes
>> to long to do anything.
>
> Hard disk speed is what helps. With my high-speed SATA RAID array I
> can build a fresh checkout and load demo data in less than 5 minutes.
> Ant clean-data plus ant load-demo takes 4 minutes. The tests take 25
> minutes - but that is mainly due to the many timing loops involved. A
> faster computer will not solve that.

Believe me, this is an *ofbiz* problem, not a hardware problem.  It
definately used to take 15m to do a full run(ant clean-all, ant
install-demo, ant run-tests).  15m was sufficient for me to start a
loop against a whole series of git commits, go away to lunch, or drive
home, and log back in to see 10 test runs all happily finished.

Things have changed in ofbiz, so that now it takes a *huge* amount of
time.  Some of that is related to a larger amount of code overflowing
the cpu's L2 cache.  Some of that is related to catalina startup, when
it tries to seed the sessions randomness, and the local machine runs
out of entropy.

I suggest that every so often software gets run on older computers;
this allows for poorly written software to be tested in a more
real-world situation, and then moree problems can be detected and fixed.

ps: I am going to switch <if> in ant *back* to javascript, *away* from
ant-contrib.  The latter is significantly slower when using a system
installed ant.

[Adam Heath-2]

On 04/18/2012 12:20 PM, Adam Heath wrote:
> Things have changed in ofbiz, so that now it takes a *huge* amount of
> time.  Some of that is related to a larger amount of code overflowing
> the cpu's L2 cache.  Some of that is related to catalina startup, when
> it tries to seed the sessions randomness, and the local machine runs
> out of entropy.

The method of placing every single theme as a *separate* catalina
webapp context is also rather stupid.  Each context drains more random
entropy.  It'd be better to have a single theme webapp, and then each
theme is a sub-folder of that.  Since themes are separate components,
this is not immediately simple; perhaps a servlet filter that
redirects paths to the appropriate theme folder.

[Jacopo Cappellato-4]

On Apr 18, 2012, at 7:20 PM, Adam Heath wrote:

> ps: I am going to switch <if> in ant *back* to javascript, *away* from
> ant-contrib.  The latter is significantly slower when using a system
> installed ant.

No please, don't do that.

Jacopo

[Jacques Le Roux]

From: "Adam Heath" <[hidden email]>

> On 04/18/2012 11:56 AM, Adrian Crum wrote:
>> On 4/18/2012 5:43 PM, Adam Heath wrote:
>>> On 04/18/2012 10:55 AM, Erwan de FERRIERES wrote:
>>>> Le 18/04/2012 17:21, Adam Heath a écrit :
>>>>> On 04/18/2012 02:57 AM, [hidden email] wrote:
>>>>>> Author: jacopoc
>>>>>> Date: Wed Apr 18 07:57:46 2012
>>>>>> New Revision: 1327411
>>>>>>
>>>>>> URL: http://svn.apache.org/viewvc?rev=1327411&view=rev
>>>>>> Log:
>>>>>> Removed bin folder and moved its content to the tools folder: this
>>>>>> was discussed extensively in the dev list recently.
>>>>>>
>>>>>> Added:
>>>>>>       ofbiz/trunk/tools/functions.sh
>>>>>>         - copied unchanged from r1327403,
>>>>>> ofbiz/trunk/bin/functions.sh
>>>>>>       ofbiz/trunk/tools/git-rebase-runner.sh
>>>>>>         - copied unchanged from r1327403,
>>>>>> ofbiz/trunk/bin/git-rebase-runner.sh
>>>>>> Removed:
>>>>>>       ofbiz/trunk/bin/
>>>>> Sorry, missed that.  I've been away from ofbiz too long.
>>>>>
>>>>>
>>>> So welcome back, then !
>>> The main reason is the ofbiz community is not paying attention to how
>>> slow the software has gotten.  It currently takes my desktop 1 hour to
>>> run a full test suite.  clean, load-demo(30m), run-tests(30m).  It
>>> used to take 12-15m.  This has caused me to not do much, as it takes
>>> to long to do anything.
>>
>> Hard disk speed is what helps. With my high-speed SATA RAID array I
>> can build a fresh checkout and load demo data in less than 5 minutes.
>> Ant clean-data plus ant load-demo takes 4 minutes. The tests take 25
>> minutes - but that is mainly due to the many timing loops involved. A
>> faster computer will not solve that.
>
> Believe me, this is an *ofbiz* problem, not a hardware problem.  It
> definately used to take 15m to do a full run(ant clean-all, ant
> install-demo, ant run-tests).  15m was sufficient for me to start a
> loop against a whole series of git commits, go away to lunch, or drive
> home, and log back in to see 10 test runs all happily finished.
>
> Things have changed in ofbiz, so that now it takes a *huge* amount of
> time.  Some of that is related to a larger amount of code overflowing
> the cpu's L2 cache.  Some of that is related to catalina startup, when
> it tries to seed the sessions randomness, and the local machine runs
> out of entropy.
>
> I suggest that every so often software gets run on older computers;
> this allows for poorly written software to be tested in a more
> real-world situation, and then moree problems can be detected and fixed.
>
> ps: I am going to switch <if> in ant *back* to javascript, *away* from
> ant-contrib.  The latter is significantly slower when using a system
> installed ant.

Maybe simply because there are more things in OFBiz. Then the current slim-down iteration should help...

Jacques

[Adam Heath-2]

On 04/18/2012 12:28 PM, Jacopo Cappellato wrote:
> On Apr 18, 2012, at 7:20 PM, Adam Heath wrote:
>
>> ps: I am going to switch <if> in ant *back* to javascript, *away* from
>> ant-contrib.  The latter is significantly slower when using a system
>> installed ant.
>
> No please, don't do that.

I've pointed out a problem, and I have a solution.  Don't just respond
saying don't do that, without a reason.  That isn't enough to stop me.

                        ant-contrib     If-ant.js
    (system-ant)
    Apache Ant version 1.8.0 compiled on March 11 2010
    time ant clean      18.131s         8.291s
    time ant build      44.072s         34.749s
    time ant build      20.228s         10.559s
    time ant clean      18.222s         8.490s
    (local-ant)
    Apache Ant(TM) version 1.8.3 compiled on February 26 2012
    time ./ant clean    3.575s          3.218s
    time ./ant build    28.985s         28.909s
    time ./ant build    5.486s          5.232s
    time ./ant clean    3.876s          3.378s

The first ant clean is run when the system is already cleaned.  The
second ant build is also run when everything is already built.  It's a
way to test what happens when nothing is done.

[Jacques Le Roux]

From: "Adam Heath" <[hidden email]>

> On 04/18/2012 12:28 PM, Jacopo Cappellato wrote:
>> On Apr 18, 2012, at 7:20 PM, Adam Heath wrote:
>>
>>> ps: I am going to switch <if> in ant *back* to javascript, *away* from
>>> ant-contrib.  The latter is significantly slower when using a system
>>> installed ant.
>>
>> No please, don't do that.
>
> I've pointed out a problem, and I have a solution.  Don't just respond
> saying don't do that, without a reason.  That isn't enough to stop me.
>
>                        ant-contrib     If-ant.js
>    (system-ant)
>    Apache Ant version 1.8.0 compiled on March 11 2010
>    time ant clean      18.131s         8.291s
>    time ant build      44.072s         34.749s
>    time ant build      20.228s         10.559s
>    time ant clean      18.222s         8.490s
>    (local-ant)
>    Apache Ant(TM) version 1.8.3 compiled on February 26 2012
>    time ./ant clean    3.575s          3.218s
>    time ./ant build    28.985s         28.909s
>    time ./ant build    5.486s          5.232s
>    time ./ant clean    3.876s          3.378s
>
> The first ant clean is run when the system is already cleaned.  The
> second ant build is also run when everything is already built.  It's a
> way to test what happens when nothing is done.

>first ant clean
ie with (system-ant)?

> second ant build
ie with (local-ant)?

I don't see much diff with (local-ant).
why not using the embedded ant?

Jacques

[Adam Heath-2]

On 04/18/2012 12:46 PM, Jacques Le Roux wrote:

> From: "Adam Heath" <[hidden email]>
>> On 04/18/2012 12:28 PM, Jacopo Cappellato wrote:
>>> On Apr 18, 2012, at 7:20 PM, Adam Heath wrote:
>>>
>>>> ps: I am going to switch <if> in ant *back* to javascript, *away*
>>>> from
>>>> ant-contrib.  The latter is significantly slower when using a system
>>>> installed ant.
>>>
>>> No please, don't do that.

You didn't give a reason as to why I shouldn't switch back.  If it's
because of things not working(at all), then list those problems.  I've
already discovered some cases of embedded jar paths in the build.xml
files that are out-of-date(I can't do cd framework/base; ../../ant
tests-cobertura, but using the system-ant does work(once I fix the
broken paths).

>> I've pointed out a problem, and I have a solution.  Don't just respond
>> saying don't do that, without a reason.  That isn't enough to stop me.
>>
>>                        ant-contrib     If-ant.js
>>    (system-ant)
>>    Apache Ant version 1.8.0 compiled on March 11 2010
>>    time ant clean      18.131s         8.291s
>>    time ant build      44.072s         34.749s
>>    time ant build      20.228s         10.559s
>>    time ant clean      18.222s         8.490s
>>    (local-ant)
>>    Apache Ant(TM) version 1.8.3 compiled on February 26 2012
>>    time ./ant clean    3.575s          3.218s
>>    time ./ant build    28.985s         28.909s
>>    time ./ant build    5.486s          5.232s
>>    time ./ant clean    3.876s          3.378s
>>
>> The first ant clean is run when the system is already cleaned.  The
>> second ant build is also run when everything is already built.  It's a
>> way to test what happens when nothing is done.
>
>> first ant clean
> ie with (system-ant)?

The first 4 runs are with the system ant.  The second 4 are with the
local ant.  notice the './' on the second set.  That's the clue.

>> second ant build
> ie with (local-ant)?
>
> I don't see much diff with (local-ant). why not using the embedded ant?

They are all slightly slower.  And that's not noise, it was consistent
in my runs.

[Jacques Le Roux]

From: "Adam Heath" <[hidden email]>

> On 04/18/2012 12:46 PM, Jacques Le Roux wrote:
>> From: "Adam Heath" <[hidden email]>
>>> On 04/18/2012 12:28 PM, Jacopo Cappellato wrote:
>>>> On Apr 18, 2012, at 7:20 PM, Adam Heath wrote:
>>>>
>>>>> ps: I am going to switch <if> in ant *back* to javascript, *away*
>>>>> from
>>>>> ant-contrib.  The latter is significantly slower when using a system
>>>>> installed ant.
>>>>
>>>> No please, don't do that.
>
> You didn't give a reason as to why I shouldn't switch back.  If it's
> because of things not working(at all), then list those problems.  I've
> already discovered some cases of embedded jar paths in the build.xml
> files that are out-of-date(I can't do cd framework/base; ../../ant
> tests-cobertura, but using the system-ant does work(once I fix the
> broken paths).
>
>>> I've pointed out a problem, and I have a solution.  Don't just respond
>>> saying don't do that, without a reason.  That isn't enough to stop me.
>>>
>>>                        ant-contrib     If-ant.js
>>>    (system-ant)
>>>    Apache Ant version 1.8.0 compiled on March 11 2010
>>>    time ant clean      18.131s         8.291s
>>>    time ant build      44.072s         34.749s
>>>    time ant build      20.228s         10.559s
>>>    time ant clean      18.222s         8.490s
>>>    (local-ant)
>>>    Apache Ant(TM) version 1.8.3 compiled on February 26 2012
>>>    time ./ant clean    3.575s          3.218s
>>>    time ./ant build    28.985s         28.909s
>>>    time ./ant build    5.486s          5.232s
>>>    time ./ant clean    3.876s          3.378s
>>>
>>> The first ant clean is run when the system is already cleaned.  The
>>> second ant build is also run when everything is already built.  It's a
>>> way to test what happens when nothing is done.
>>
>>> first ant clean
>> ie with (system-ant)?
>
> The first 4 runs are with the system ant.  The second 4 are with the
> local ant.  notice the './' on the second set.  That's the clue.
>
>>> second ant build
>> ie with (local-ant)?
>>
>> I don't see much diff with (local-ant). why not using the embedded ant?
>
> They are all slightly slower.  And that's not noise, it was consistent
> in my runs.

I'd not worry about so low diff. respectively:
+11 %
+0.2 %
+4 %
+14 %
not significant enough to take action IMO (don't fix when it works)

I don't remember why I prefered ant-contrib,but I can digg deeper...

Jacques

[Erwan de FERRIERES]

Le 18/04/2012 18:43, Adam Heath a écrit :

> On 04/18/2012 10:55 AM, Erwan de FERRIERES wrote:
>> Le 18/04/2012 17:21, Adam Heath a écrit :
>>> On 04/18/2012 02:57 AM, [hidden email] wrote:
>>>> Author: jacopoc
>>>> Date: Wed Apr 18 07:57:46 2012
>>>> New Revision: 1327411
>>>>
>>>> URL: http://svn.apache.org/viewvc?rev=1327411&view=rev
>>>> Log:
>>>> Removed bin folder and moved its content to the tools folder: this
>>>> was discussed extensively in the dev list recently.
>>>>
>>>> Added:
>>>>       ofbiz/trunk/tools/functions.sh
>>>>         - copied unchanged from r1327403, ofbiz/trunk/bin/functions.sh
>>>>       ofbiz/trunk/tools/git-rebase-runner.sh
>>>>         - copied unchanged from r1327403,
>>>> ofbiz/trunk/bin/git-rebase-runner.sh
>>>> Removed:
>>>>       ofbiz/trunk/bin/
>>>
>>> Sorry, missed that.  I've been away from ofbiz too long.
>>>
>>>
>> So welcome back, then !
>
> The main reason is the ofbiz community is not paying attention to how
> slow the software has gotten.  It currently takes my desktop 1 hour to
> run a full test suite.  clean, load-demo(30m), run-tests(30m).  It
> used to take 12-15m.  This has caused me to not do much, as it takes
> to long to do anything.
>
> The issue that helps the most is getting a larger l2-cache on the cpu.
>   The load-demo time is greatly affected by derby being *way* to large,
> code-size wise.
>
> I've started doing work now, by using a virtualized instance that has
> more l2-cache, but the problem still remains.
>

Hi,

I had better stats with the new ant 1.8.3. But the times you have are
really impressive !
can't explain why...




--
Erwan de FERRIERES
www.nereide.biz

[Scott Gray-2]

On 19/04/2012, at 5:54 AM, Adam Heath wrote:

> On 04/18/2012 12:46 PM, Jacques Le Roux wrote:
>> From: "Adam Heath" <[hidden email]>
>>> On 04/18/2012 12:28 PM, Jacopo Cappellato wrote:
>>>> On Apr 18, 2012, at 7:20 PM, Adam Heath wrote:
>>>>
>>>>> ps: I am going to switch <if> in ant *back* to javascript, *away*
>>>>> from
>>>>> ant-contrib.  The latter is significantly slower when using a system
>>>>> installed ant.
>>>>
>>>> No please, don't do that.
>
> You didn't give a reason as to why I shouldn't switch back.  If it's
> because of things not working(at all), then list those problems.  I've
> already discovered some cases of embedded jar paths in the build.xml
> files that are out-of-date(I can't do cd framework/base; ../../ant
> tests-cobertura, but using the system-ant does work(once I fix the
> broken paths).

Have you considered working with the ant project at all to fix the issue?  The problem we have and I think we want to address is simply a case of way too much code in this project trying to do way too many things and being maintained by nowhere near enough people with the expertise to do so. We're attempting to take a good hard look at where we can pass the responsibility for maintaining that functionality off to more appropriate parties.  Unless there is a very good reason for not contributing improvements back to ant then I can't see why it wouldn't more sense to do so.

>>> I've pointed out a problem, and I have a solution.  Don't just respond
>>> saying don't do that, without a reason.  That isn't enough to stop me.
>>>
>>>                       ant-contrib     If-ant.js
>>>   (system-ant)
>>>   Apache Ant version 1.8.0 compiled on March 11 2010
>>>   time ant clean      18.131s         8.291s
>>>   time ant build      44.072s         34.749s
>>>   time ant build      20.228s         10.559s
>>>   time ant clean      18.222s         8.490s
>>>   (local-ant)
>>>   Apache Ant(TM) version 1.8.3 compiled on February 26 2012
>>>   time ./ant clean    3.575s          3.218s
>>>   time ./ant build    28.985s         28.909s
>>>   time ./ant build    5.486s          5.232s
>>>   time ./ant clean    3.876s          3.378s
>>>
>>> The first ant clean is run when the system is already cleaned.  The
>>> second ant build is also run when everything is already built.  It's a
>>> way to test what happens when nothing is done.
>>
>>> first ant clean
>> ie with (system-ant)?
>
> The first 4 runs are with the system ant.  The second 4 are with the
> local ant.  notice the './' on the second set.  That's the clue.
>
>>> second ant build
>> ie with (local-ant)?
>>
>> I don't see much diff with (local-ant). why not using the embedded ant?
>
> They are all slightly slower.  And that's not noise, it was consistent
> in my runs.

[Rajbir Saini]

We resolved per theme web app problem by moving the them header/footer
to CMS. In multi-tenant e-commerce applications it is possible to have
100's of themes and it is not possible to create web application for
each theme. What is needed is create different header/footer content for
each theme and create the theme data accordingly.

Raj

On Wednesday 18 April 2012 10:54 PM, Adam Heath wrote:

> On 04/18/2012 12:20 PM, Adam Heath wrote:
>> Things have changed in ofbiz, so that now it takes a *huge* amount of
>> time.  Some of that is related to a larger amount of code overflowing
>> the cpu's L2 cache.  Some of that is related to catalina startup, when
>> it tries to seed the sessions randomness, and the local machine runs
>> out of entropy.
> The method of placing every single theme as a *separate* catalina
> webapp context is also rather stupid.  Each context drains more random
> entropy.  It'd be better to have a single theme webapp, and then each
> theme is a sub-folder of that.  Since themes are separate components,
> this is not immediately simple; perhaps a servlet filter that
> redirects paths to the appropriate theme folder.
>
>

[Jacopo Cappellato-4]

On Apr 18, 2012, at 7:37 PM, Adam Heath wrote:

> On 04/18/2012 12:28 PM, Jacopo Cappellato wrote:
>> On Apr 18, 2012, at 7:20 PM, Adam Heath wrote:
>>
>>> ps: I am going to switch <if> in ant *back* to javascript, *away* from
>>> ant-contrib.  The latter is significantly slower when using a system
>>> installed ant.
>>
>> No please, don't do that.
>
> I've pointed out a problem, and I have a solution.  Don't just respond
> saying don't do that, without a reason.  That isn't enough to stop me.

I thought that all the recent threads about code refactoring, moving out custom code etc... were enough to motivate my response; but it seems that you ignored all of them and now pretend a response from me, for such a trivial and small topic.
The reason is that we want to keep code as clean as possible, as slim as possible (ok, I know it looks silly if you look at the code now... but we are determined to work on this): using an external tool for this instead of custom code is a step in this direction.
The stats below show that the problem is trivial: it is only relevant when you use your own ant that is not what we recommend; and it is something that should be fixed with the ant community: in fact we want to increase the cooperation with other communities (especially if they are in the ASF world) rather than blobbing everything into OFBiz.

Jacopo

>
>                        ant-contrib     If-ant.js
>    (system-ant)
>    Apache Ant version 1.8.0 compiled on March 11 2010
>    time ant clean      18.131s         8.291s
>    time ant build      44.072s         34.749s
>    time ant build      20.228s         10.559s
>    time ant clean      18.222s         8.490s
>    (local-ant)
>    Apache Ant(TM) version 1.8.3 compiled on February 26 2012
>    time ./ant clean    3.575s          3.218s
>    time ./ant build    28.985s         28.909s
>    time ./ant build    5.486s          5.232s
>    time ./ant clean    3.876s          3.378s
>
> The first ant clean is run when the system is already cleaned.  The
> second ant build is also run when everything is already built.  It's a
> way to test what happens when nothing is done.

[Jacopo Cappellato-4]

On Apr 18, 2012, at 6:55 PM, Jacopo Cappellato wrote:

> On Apr 18, 2012, at 6:43 PM, Adam Heath wrote:
>
>> It currently takes my desktop 1 hour to
>> run a full test suite.  clean, load-demo(30m), run-tests(30m).  It
>> used to take 12-15m.
>
> wow, it takes me 2-3 minutes in my laptop.
>
> Jacopo
>

To be more precise:
(less than) 3 minutes for load-demo
4 minutes and 3 seconds for run-tests

Jacopo

[Jacques Le Roux]

OK seems after all that I need a brand new CPU, hence a new box...
I must say with the SSDs when running run-tests my CPU is always at 100% (I put the run-tests process priority level to idle to be
able to continue other tasks)
Do you have a SSD in your laptop Jacopo?

Jacques

From: "Jacopo Cappellato" <[hidden email]>

> On Apr 18, 2012, at 6:55 PM, Jacopo Cappellato wrote:
>
>> On Apr 18, 2012, at 6:43 PM, Adam Heath wrote:
>>
>>> It currently takes my desktop 1 hour to
>>> run a full test suite.  clean, load-demo(30m), run-tests(30m).  It
>>> used to take 12-15m.
>>
>> wow, it takes me 2-3 minutes in my laptop.
>>
>> Jacopo
>>
>
>
> To be more precise:
> (less than) 3 minutes for load-demo
> 4 minutes and 3 seconds for run-tests
>
> Jacopo
>