demo server performance
Locked
123
123
 
|
there is a thread on the user ML about the demo being slow.
I would think that would be a high priority for all those that commit and make changes to ofbiz. after all what good is all this stuff if it can't be used. I brought down the demo trunk by accessing with seperate requests at one time, as I stated on the user ml. lets focus on real problems. |
|
|
Everybody works on the areas of the system that are important to them, I suggest you do the same.
The demo server is under-resourced so of course you're going to be able to bring it down if you try to, my suggestion is that you don't try to. Regards Scott HotWax Media http://www.hotwaxmedia.com On 10/12/2010, at 10:32 AM, BJ Freeman wrote: > there is a thread on the user ML about the demo being slow. > I would think that would be a high priority for all those that commit and make changes to ofbiz. > after all what good is all this stuff if it can't be used. > I brought down the demo trunk by accessing with seperate requests at one time, as I stated on the user ml. > > lets focus on real problems. > > > |
smime.p7s (3K) |
|
In reply to this post by BJ Freeman
On Dec 9, 2010, at 1:32 PM, BJ Freeman wrote: > there is a thread on the user ML about the demo being slow. > I would think that would be a high priority for all those that commit and make changes to ofbiz. > after all what good is all this stuff if it can't be used. > I brought down the demo trunk by accessing with seperate requests at one time, as I stated on the user ml. > > lets focus on real problems. The real problem is: real problems according to who/what? Don't make the mistake of thinking that problems on the demo server mean that real-world users with production instances are having the same problems. Also, consider that many of the features (like the WebTools -> Artifact Info stuff) are NOT meant to be run on production systems and doing so is guaranteed to use system resources in a wasteful way. The main way that things get fixed is by a "real-world" user dedicating resources to fixing things that are important to their use of the system, and if you look at the commit logs you'll see those kinds of fixes (and/or improvements) coming in all the time. That's what drives the project. Scott mentioned that the demo server is under-resourced, and that is true of hardware resources AND human resources. If not enough people care about it, there is no means of force or incentive from the project itself to get it done. BTW BJ, why in your message did you limit the people who should do something about this to "all those that commit and make changes to ofbiz"? Or did you mean that more generally, like anyone who changes OFBiz, which would include you too? If that's not what you meant, then would you consider yourself to be in the category of person that believes that a voluntary gift by someone obligates them to future involuntary gifts? And what will you do if they stop giving and your entitlements are gone? -David |
|
|
In reply to this post by Scott Gray-2
Thanks for you view on my motives.
From what Jacques states the server has max hardware resources. so what resources are you referring to? I since I have a similar server running more than what Jacques has stated, and it runs, I am at a loss on how to work on the ofbiz demo. I have been focused as much as I am allowed on this for almost a year. considering posting five urls at the same time should not effect a server, I don't see that as testing the limits of the server. Scott Gray sent the following on 12/9/2010 1:47 PM: > Everybody works on the areas of the system that are important to them, I suggest you do the same. > > The demo server is under-resourced so of course you're going to be able to bring it down if you try to, my suggestion is that you don't try to. > > Regards > Scott > > HotWax Media > http://www.hotwaxmedia.com > > On 10/12/2010, at 10:32 AM, BJ Freeman wrote: > >> there is a thread on the user ML about the demo being slow. >> I would think that would be a high priority for all those that commit and make changes to ofbiz. >> after all what good is all this stuff if it can't be used. >> I brought down the demo trunk by accessing with seperate requests at one time, as I stated on the user ml. >> >> lets focus on real problems. >> >> >> > |
|
|
In reply to this post by David E. Jones-2
I have, what I consider, not only production but demo and test instances
on a server. an they work so I agree. BTW I can use the artifacts on my production so that is not a consideration. I mean those that review the code that gets committed and have access to the demo server. I answer the resources in my reply to scott. If everyone stop contributing the way they do(little or no intensive testing, and upgrade paths), maybe I could get release stable. So I don't see the "gifts" as that. that is slowly being taken care of as my resources are gaining. ======================== BJ Freeman Strategic Power Office with Supplier Automation <http://www.businessesnetwork.com/automation/viewforum.php?f=52> Specialtymarket.com <http://www.specialtymarket.com/> Systems Integrator-- Glad to Assist Chat Y! messenger: bjfr33man David E Jones sent the following on 12/9/2010 2:23 PM: > > On Dec 9, 2010, at 1:32 PM, BJ Freeman wrote: > >> there is a thread on the user ML about the demo being slow. >> I would think that would be a high priority for all those that commit and make changes to ofbiz. >> after all what good is all this stuff if it can't be used. >> I brought down the demo trunk by accessing with seperate requests at one time, as I stated on the user ml. >> >> lets focus on real problems. > > The real problem is: real problems according to who/what? > > Don't make the mistake of thinking that problems on the demo server mean that real-world users with production instances are having the same problems. Also, consider that many of the features (like the WebTools -> Artifact Info stuff) are NOT meant to be run on production systems and doing so is guaranteed to use system resources in a wasteful way. > > The main way that things get fixed is by a "real-world" user dedicating resources to fixing things that are important to their use of the system, and if you look at the commit logs you'll see those kinds of fixes (and/or improvements) coming in all the time. That's what drives the project. > > Scott mentioned that the demo server is under-resourced, and that is true of hardware resources AND human resources. If not enough people care about it, there is no means of force or incentive from the project itself to get it done. > > BTW BJ, why in your message did you limit the people who should do something about this to "all those that commit and make changes to ofbiz"? Or did you mean that more generally, like anyone who changes OFBiz, which would include you too? > > If that's not what you meant, then would you consider yourself to be in the category of person that believes that a voluntary gift by someone obligates them to future involuntary gifts? And what will you do if they stop giving and your entitlements are gone? > > -David > > |
|
|
In reply to this post by BJ Freeman
On 12/09/2010 05:01 PM, BJ Freeman wrote:
> Thanks for you view on my motives. > From what Jacques states the server has max hardware resources. > so what resources are you referring to? > I since I have a similar server running more than what Jacques has > stated, and it runs, I am at a loss on how to work on the ofbiz demo. > I have been focused as much as I am allowed on this for almost a year. > considering posting five urls at the same time should not effect a > server, I don't see that as testing the limits of the server. What urls? What actions are you performing, and what do you expect to happen? Details, please. 'max hardware' to me means that it has the most resources that are available. It most definately does not mean that it is running on the fastest computer known to man. Plus, it is not tuned to for its installation. Installing a production system for a client requires tuning tons of different knobs. For each install, those knobs will be different. It does not make sense to change the sane defaults in ofbiz to something that is for one particular install(apache demo installs). As David said, this project is just a bunch of volunteers. If you see a problem, and no one steps up to resolve it(or, at the least, investigate), then it falls on the reporter to do the work. If that doesn't happen, then I guess nothing will be finished. But you can't force anyone in this project to do anything. The best you could do(if I could borrow the terminology) is to show the business case for why something should be better, and get others to be excited about fixing it. Then you can just sit back and watch others do work for you. |
Administrator
|
In reply to this post by BJ Freeman
I have spent a lot of time (I mean in respect of my free time) this last days to understand the problems.
It appears that removing the help was a great relief for our demo sever. For few hours now we are running with trunk: -Xms128M -Xmx768M -XX:MaxPermSize=192m (so max seems 768+192=960MB but actually it's more) branch9: -Xms128M -Xmx512M For instance now we have Mem: 2573924k total, 2159888k used, 414036k free, 53672k buffers Swap: 1502036k total, 50676k used, 1451360k free, 438000k cached PID USER PR NI VIRT RES SHR %CPU %MEM trunk 14896 ofbiz 20 0 1377m 753m 7956 0.3 30.0 branch9 18147 ofbiz 20 0 918m 670m 13m 0.7 26.7 As you can see at some stage we reach more than 960MB for the trunk (1377 max, which is approx but anyway) The main points: * We have still around 400MB free, but I suppose it will be less just before the 24h reload) * We have anymore CPU running always near 100%, for instance right now PID USER PR NI VIRT RES SHR %CPU %MEM TIME+ COMMAND 14896 ofbiz 20 0 1377m 757m 7968 29.7 30.2 19:57.63 java -Xms128M -Xmx768M -XX:MaxPermSize=192m 18147 ofbiz 20 0 918m 671m 13m 22.4 26.7 14:23.55 java -Xms128M -Xmx512M I will wait some days and, if things continue to go well, will re-use more memory for our 2 processes. But I know there are other problems... Like David and Scott said if people are using the Artifact Info or other gluttonous features (Birts?) we will be in trouble with our memory quota. So if such things come back in the future I will suggest to prevent users to use them on the demo server... For the real problems, I think we should focus on fixing the online Help feature. It seems that this isues is something relatively new and a disect should help (I use this word because it's convenient, on my side I simply use dichotomic tests with svn but I have bigger fish to fry for now, that's why I have deactivated it). I think it's not more than few days (weeks?), help appreciated... Thanks Jacques From: "BJ Freeman" <[hidden email]> > there is a thread on the user ML about the demo being slow. > I would think that would be a high priority for all those that commit and make changes to ofbiz. > after all what good is all this stuff if it can't be used. > I brought down the demo trunk by accessing with seperate requests at one time, as I stated on the user ml. > > lets focus on real problems. > > > |
|
|
In reply to this post by BJ Freeman
On 12/09/2010 05:11 PM, BJ Freeman wrote:
> [snip, just want to talk about the next paragraph] > If everyone stop contributing the way they do(little or no intensive > testing, and upgrade paths), maybe I could get release stable. So I > don't see the "gifts" as that. What do you mean, getting a release stable? Are you deploying new applications on trunk, and then after the deployment, continuing to upgrade to the latest trunk each time, in a production environment? If so, then don't do that. In all the various open source projects I have been involved with, *none* have ever done full upgrade testing, full system regression, on trunk. Only when the set of features stablizes, and a real release is iniment, does final upgrade testing occur, and release notes get finalized. A feature added to trunk during a development cycle may end up getting completely rewritten, or removed, before the next stable release comes out. If we follow your design practice, then every single change will have to come with full deprecation of old features, full upgrade support, and nothing will actually get implemented. Here at brainfood, our current ofbiz deployments are based on svn 902021. There was nothing special about that particular version. I just happened to have time to create a new ofbiz.deb package(we use debian internally). Since then, there have been 447 commits to our local package, and 42 separate package releases. 99% of those changes are backports(actually, we use git, so they are called cherry-picks) directly from trunk. I've been able to do this all on my own, a single person, in addition to my other duties. It has not been a huge sap on my personal resources. |
|
|
In reply to this post by Jacques Le Roux
On 12/09/2010 05:18 PM, Jacques Le Roux wrote:
> I have spent a lot of time (I mean in respect of my free time) this last > days to understand the problems. > It appears that removing the help was a great relief for our demo sever. > > For few hours now we are running with > > trunk: -Xms128M -Xmx768M -XX:MaxPermSize=192m (so max seems > 768+192=960MB but actually it's more) > branch9: -Xms128M -Xmx512M > > For instance now we have > Mem: 2573924k total, 2159888k used, 414036k free, 53672k buffers > Swap: 1502036k total, 50676k used, 1451360k free, 438000k cached > > PID USER PR NI VIRT RES SHR %CPU %MEM > trunk 14896 ofbiz 20 0 1377m 753m 7956 0.3 30.0 > branch9 18147 ofbiz 20 0 918m 670m 13m 0.7 26.7 > > As you can see at some stage we reach more than 960MB for the trunk > (1377 max, which is approx but anyway) > > The main points: > * We have still around 400MB free, but I suppose it will be less just > before the 24h reload) > * We have anymore CPU running always near 100%, for instance right now > PID USER PR NI VIRT RES SHR %CPU %MEM TIME+ COMMAND > 14896 ofbiz 20 0 1377m 757m 7968 29.7 30.2 19:57.63 java -Xms128M > -Xmx768M -XX:MaxPermSize=192m > 18147 ofbiz 20 0 918m 671m 13m 22.4 26.7 14:23.55 java -Xms128M -Xmx512M > > > I will wait some days and, if things continue to go well, will re-use > more memory for our 2 processes. But I know there are other > problems... > > Like David and Scott said if people are using the Artifact Info or other > gluttonous features (Birts?) we will be in trouble with our > memory quota. So if such things come back in the future I will suggest > to prevent users to use them on the demo server... > > For the real problems, I think we should focus on fixing the online Help > feature. It seems that this isues is something relatively > new and a disect should help (I use this word because it's convenient, > on my side I simply use dichotomic tests with svn but I have > bigger fish to fry for now, that's why I have deactivated it). I think > it's not more than few days (weeks?), help appreciated... Hate to disappoint, but all those memory stats you posted are completely useless for actually tracking down what java is doing. You need to become friends with jmap, jhat(both standard jdk tools), and ibm's heap analyzer. Plus, sending the QUIT signal to the java process. In all honesty, I'm going to go out on a limb here and say the higher memory requirements of newer ofbiz is due to converting tons of code to groovy. With it as a simple-method, or a bsh, both would end up using heap, as they are interpetted. java or groovy get compiled to bytecode, which ends up being allocated in the permgen area, which might also get jit compiled. So, permgen needs to increase. |
|
Administrator
|
In reply to this post by BJ Freeman
From: "BJ Freeman" <[hidden email]>
> Thanks for you view on my motives. > From what Jacques states the server has max hardware resources. > so what resources are you referring to? > I since I have a similar server running more than what Jacques has stated, and it runs, I am at a loss on how to work on the ofbiz > demo. > I have been focused as much as I am allowed on this for almost a year. > considering posting five urls at the same time should not effect a server, I don't see that as testing the limits of the server. Which URLs? It really depends on them... Artifact Info, Entity Maintenance, Label Manager, etc. are good culprits... This does not mean that we can't use Entity Maintenance on the demo server, nor even Artifact Info. But it depends on the number of people which are using them at the same time. And when it's down, it's down: you will have to wait a good soul (not sleeping, like me in some mins) to reload the demo instance... Webtools are not all days tools for a production server... I will suggest to use rather such tools locally... Does it make sense? Thanks Jacques > > Scott Gray sent the following on 12/9/2010 1:47 PM: >> Everybody works on the areas of the system that are important to them, I suggest you do the same. >> >> The demo server is under-resourced so of course you're going to be able to bring it down if you try to, my suggestion is that you >> don't try to. >> >> Regards >> Scott >> >> HotWax Media >> http://www.hotwaxmedia.com >> >> On 10/12/2010, at 10:32 AM, BJ Freeman wrote: >> >>> there is a thread on the user ML about the demo being slow. >>> I would think that would be a high priority for all those that commit and make changes to ofbiz. >>> after all what good is all this stuff if it can't be used. >>> I brought down the demo trunk by accessing with seperate requests at one time, as I stated on the user ml. >>> >>> lets focus on real problems. >>> >>> >>> >> > |
|
|
In reply to this post by Adam Heath-2
when I restart my client firefox brings up all the pages from last session.
the following URl are accessed from different tabs. I expect them to come up to the login screen except for the ecommerce. many times I will get problem loading http://demo-trunk.ofbiz.apache.org:8080/ecommerce/control/main https://demo-trunk.ofbiz.apache.org/accounting/control/FindBillingAccount?partyId=DemoCustomer https://demo-trunk.ofbiz.apache.org/webtools/control/login https://demo-trunk.ofbiz.apache.org/partymgr/control/editpartygroup?partyId=DemoSupplier https://demo-trunk.ofbiz.apache.org/webtools/control/FindGeneric?entityName=Product&find=true&VIEW_SIZE=50&VIEW_INDEX=0 lately when they load http://monitoring.apache.org/status/ shows everthing is ok I then load the above URLs and get a warning then flap then shutdown. I can not duplicate the above on my demo-trunk on my server. ========================= BJ Freeman Strategic Power Office with Supplier Automation <http://www.businessesnetwork.com/automation/viewforum.php?f=52> Specialtymarket.com <http://www.specialtymarket.com/> Systems Integrator-- Glad to Assist Chat Y! messenger: bjfr33man Adam Heath sent the following on 12/9/2010 3:13 PM: > On 12/09/2010 05:01 PM, BJ Freeman wrote: >> Thanks for you view on my motives. >> From what Jacques states the server has max hardware resources. >> so what resources are you referring to? >> I since I have a similar server running more than what Jacques has >> stated, and it runs, I am at a loss on how to work on the ofbiz demo. >> I have been focused as much as I am allowed on this for almost a year. >> considering posting five urls at the same time should not effect a >> server, I don't see that as testing the limits of the server. > > What urls? What actions are you performing, and what do you expect to > happen? Details, please. > > 'max hardware' to me means that it has the most resources that are > available. It most definately does not mean that it is running on the > fastest computer known to man. > > Plus, it is not tuned to for its installation. Installing a production > system for a client requires tuning tons of different knobs. For each > install, those knobs will be different. It does not make sense to change > the sane defaults in ofbiz to something that is for one particular > install(apache demo installs). > > As David said, this project is just a bunch of volunteers. If you see a > problem, and no one steps up to resolve it(or, at the least, > investigate), then it falls on the reporter to do the work. If that > doesn't happen, then I guess nothing will be finished. But you can't > force anyone in this project to do anything. > > The best you could do(if I could borrow the terminology) is to show the > business case for why something should be better, and get others to be > excited about fixing it. Then you can just sit back and watch others do > work for you. > |
|
Administrator
|
In reply to this post by Adam Heath-2
From: "Adam Heath" <[hidden email]>
> On 12/09/2010 05:18 PM, Jacques Le Roux wrote: >> I have spent a lot of time (I mean in respect of my free time) this last >> days to understand the problems. >> It appears that removing the help was a great relief for our demo sever. >> >> For few hours now we are running with >> >> trunk: -Xms128M -Xmx768M -XX:MaxPermSize=192m (so max seems >> 768+192=960MB but actually it's more) >> branch9: -Xms128M -Xmx512M >> >> For instance now we have >> Mem: 2573924k total, 2159888k used, 414036k free, 53672k buffers >> Swap: 1502036k total, 50676k used, 1451360k free, 438000k cached >> >> PID USER PR NI VIRT RES SHR %CPU %MEM >> trunk 14896 ofbiz 20 0 1377m 753m 7956 0.3 30.0 >> branch9 18147 ofbiz 20 0 918m 670m 13m 0.7 26.7 >> >> As you can see at some stage we reach more than 960MB for the trunk >> (1377 max, which is approx but anyway) >> >> The main points: >> * We have still around 400MB free, but I suppose it will be less just >> before the 24h reload) >> * We have anymore CPU running always near 100%, for instance right now >> PID USER PR NI VIRT RES SHR %CPU %MEM TIME+ COMMAND >> 14896 ofbiz 20 0 1377m 757m 7968 29.7 30.2 19:57.63 java -Xms128M >> -Xmx768M -XX:MaxPermSize=192m >> 18147 ofbiz 20 0 918m 671m 13m 22.4 26.7 14:23.55 java -Xms128M -Xmx512M >> >> >> I will wait some days and, if things continue to go well, will re-use >> more memory for our 2 processes. But I know there are other >> problems... >> >> Like David and Scott said if people are using the Artifact Info or other >> gluttonous features (Birts?) we will be in trouble with our >> memory quota. So if such things come back in the future I will suggest >> to prevent users to use them on the demo server... >> >> For the real problems, I think we should focus on fixing the online Help >> feature. It seems that this isues is something relatively >> new and a disect should help (I use this word because it's convenient, >> on my side I simply use dichotomic tests with svn but I have >> bigger fish to fry for now, that's why I have deactivated it). I think >> it's not more than few days (weeks?), help appreciated... > > Hate to disappoint, but all those memory stats you posted are completely useless for actually tracking down what java is doing. > > You need to become friends with jmap, jhat(both standard jdk tools), and ibm's heap analyzer. Plus, sending the QUIT signal to > the java process. Yes I know, this is only to give a general information about what's going on on the server. As I have already wrote I'm actually using mat http://www.eclipse.org/mat/ behind the scene I'm also using -XX:+HeapDumpOnOutOfMemoryError but most of the time get rather out of swap issues when crashing, hard to trace... One way would be mod_log_forensic... if someone wants to help... > In all honesty, I'm going to go out on a limb here and say the higher memory requirements of newer ofbiz is due to converting tons > of code to groovy. With it as a simple-method, or a bsh, both would end up using heap, as they are interpetted. java or groovy > get compiled to bytecode, which ends up being allocated in the permgen area, which might also get jit compiled. So, permgen needs > to increase. It does not seem that we have permgen issues. It's not yet clear, but for those interested I could move hprof files from demo roots to bigfiles dir... Thanks Jacques |
|
|
In reply to this post by BJ Freeman
On 12/09/2010 05:31 PM, BJ Freeman wrote:
> when I restart my client firefox brings up all the pages from last > session. > the following URl are accessed from different tabs. > I expect them to come up to the login screen except for the ecommerce. > many times I will get > problem loading > http://demo-trunk.ofbiz.apache.org:8080/ecommerce/control/main > https://demo-trunk.ofbiz.apache.org/accounting/control/FindBillingAccount?partyId=DemoCustomer > > https://demo-trunk.ofbiz.apache.org/webtools/control/login > https://demo-trunk.ofbiz.apache.org/partymgr/control/editpartygroup?partyId=DemoSupplier > > https://demo-trunk.ofbiz.apache.org/webtools/control/FindGeneric?entityName=Product&find=true&VIEW_SIZE=50&VIEW_INDEX=0 So, 5 requests issued at all once. Should be easy to duplicate that. Start up a local ofbiz, hit then right after startup, then hit them again after the cache(s) are primed. > lately when they load > http://monitoring.apache.org/status/ > shows everthing is ok > I then load the above URLs and get a warning then flap then shutdown. That status link can't be used for debugging these kinds of problems. It's the as saying "I found a problem in a groovy file, but the machine pings, so I'm at a loss as to what is going on." |
|
Administrator
|
In reply to this post by BJ Freeman
From: "BJ Freeman" <[hidden email]>
> when I restart my client firefox brings up all the pages from last session. > the following URl are accessed from different tabs. > I expect them to come up to the login screen except for the ecommerce. > many times I will get > problem loading > http://demo-trunk.ofbiz.apache.org:8080/ecommerce/control/main > https://demo-trunk.ofbiz.apache.org/accounting/control/FindBillingAccount?partyId=DemoCustomer > https://demo-trunk.ofbiz.apache.org/webtools/control/login > https://demo-trunk.ofbiz.apache.org/partymgr/control/editpartygroup?partyId=DemoSupplier > https://demo-trunk.ofbiz.apache.org/webtools/control/FindGeneric?entityName=Product&find=true&VIEW_SIZE=50&VIEW_INDEX=0 What kinds of problems? > lately when they load > http://monitoring.apache.org/status/ > shows everthing is ok > I then load the above URLs and get a warning then flap then shutdown. With the above URLs all should go smoothly, I'm surprised... I will test tomorrow... Jacques > I can not duplicate the above on my demo-trunk on my server. > > ========================= > > BJ Freeman > Strategic Power Office with Supplier Automation <http://www.businessesnetwork.com/automation/viewforum.php?f=52> > Specialtymarket.com <http://www.specialtymarket.com/> > Systems Integrator-- Glad to Assist > > Chat Y! messenger: bjfr33man > > > Adam Heath sent the following on 12/9/2010 3:13 PM: >> On 12/09/2010 05:01 PM, BJ Freeman wrote: >>> Thanks for you view on my motives. >>> From what Jacques states the server has max hardware resources. >>> so what resources are you referring to? >>> I since I have a similar server running more than what Jacques has >>> stated, and it runs, I am at a loss on how to work on the ofbiz demo. >>> I have been focused as much as I am allowed on this for almost a year. >>> considering posting five urls at the same time should not effect a >>> server, I don't see that as testing the limits of the server. >> >> What urls? What actions are you performing, and what do you expect to >> happen? Details, please. >> >> 'max hardware' to me means that it has the most resources that are >> available. It most definately does not mean that it is running on the >> fastest computer known to man. >> >> Plus, it is not tuned to for its installation. Installing a production >> system for a client requires tuning tons of different knobs. For each >> install, those knobs will be different. It does not make sense to change >> the sane defaults in ofbiz to something that is for one particular >> install(apache demo installs). >> >> As David said, this project is just a bunch of volunteers. If you see a >> problem, and no one steps up to resolve it(or, at the least, >> investigate), then it falls on the reporter to do the work. If that >> doesn't happen, then I guess nothing will be finished. But you can't >> force anyone in this project to do anything. >> >> The best you could do(if I could borrow the terminology) is to show the >> business case for why something should be better, and get others to be >> excited about fixing it. Then you can just sit back and watch others do >> work for you. >> > |
|
|
In reply to this post by Adam Heath-2
all my changes are in seperate components.
currently my attention is 10.4 branch, which I keep updating in a test instance. 1)load and configure my componets 2)ant run-install check logs and operation related to my components. what usually happens is I get log errors do to changes that are not compatible from 9.04. 3)connect to postgresql 4)load data from production server into this test instance. set entityengine so will not update db. 5)run code so get logs reports of data changes 6)see if any effect my components 7)got through the compoents to change. 8)ant clean-all 9)clean the postresql and turn back on add tables. 10)run migration of production data to changed db for 10.4 first time workeffort was a few manweeks. each subsequence update of the 10.4 does not exceed a manweek. ========================= BJ Freeman Strategic Power Office with Supplier Automation <http://www.businessesnetwork.com/automation/viewforum.php?f=52> Specialtymarket.com <http://www.specialtymarket.com/> Systems Integrator-- Glad to Assist Chat Y! messenger: bjfr33man Adam Heath sent the following on 12/9/2010 3:22 PM: > On 12/09/2010 05:11 PM, BJ Freeman wrote: > > [snip, just want to talk about the next paragraph] > >> If everyone stop contributing the way they do(little or no intensive >> testing, and upgrade paths), maybe I could get release stable. So I >> don't see the "gifts" as that. > > What do you mean, getting a release stable? Are you deploying new > applications on trunk, and then after the deployment, continuing to > upgrade to the latest trunk each time, in a production environment? If > so, then don't do that. > > In all the various open source projects I have been involved with, > *none* have ever done full upgrade testing, full system regression, on > trunk. Only when the set of features stablizes, and a real release is > iniment, does final upgrade testing occur, and release notes get finalized. > > A feature added to trunk during a development cycle may end up getting > completely rewritten, or removed, before the next stable release comes > out. If we follow your design practice, then every single change will > have to come with full deprecation of old features, full upgrade > support, and nothing will actually get implemented. > > Here at brainfood, our current ofbiz deployments are based on svn > 902021. There was nothing special about that particular version. I just > happened to have time to create a new ofbiz.deb package(we use debian > internally). Since then, there have been 447 commits to our local > package, and 42 separate package releases. 99% of those changes are > backports(actually, we use git, so they are called cherry-picks) > directly from trunk. I've been able to do this all on my own, a single > person, in addition to my other duties. It has not been a huge sap on my > personal resources. > |
|
|
On 12/09/2010 05:50 PM, BJ Freeman wrote:
> all my changes are in seperate components. > currently my attention is 10.4 branch, which I keep updating in a test > instance. > 1)load and configure my componets > 2)ant run-install > check logs and operation related to my components. > what usually happens is I get log errors do to changes that are not > compatible from 9.04. > 3)connect to postgresql > 4)load data from production server into this test instance. > set entityengine so will not update db. > 5)run code so get logs reports of data changes > 6)see if any effect my components > 7)got through the compoents to change. > 8)ant clean-all > 9)clean the postresql and turn back on add tables. > 10)run migration of production data to changed db for 10.4 > > first time workeffort was a few manweeks. > each subsequence update of the 10.4 does not exceed a manweek. Interesting. Yes, I agree, a defined release branch in ofbiz(10.4 in this case) should have such upgrade support from one minor point release to another. However, there actually hasn't been a real release of that branch, so, ... |
|
|
In reply to this post by Adam Heath-2
all the status page does is give my an indicatioh if the demo is healthy
before I start I have done it enough time to figure out that it probably not other influences. Since I don't have access to the logs on the demo-trunk, I can not do further debug. ========================= BJ Freeman Strategic Power Office with Supplier Automation <http://www.businessesnetwork.com/automation/viewforum.php?f=52> Specialtymarket.com <http://www.specialtymarket.com/> Systems Integrator-- Glad to Assist Chat Y! messenger: bjfr33man Adam Heath sent the following on 12/9/2010 3:43 PM: > On 12/09/2010 05:31 PM, BJ Freeman wrote: >> when I restart my client firefox brings up all the pages from last >> session. >> the following URl are accessed from different tabs. >> I expect them to come up to the login screen except for the ecommerce. >> many times I will get >> problem loading >> http://demo-trunk.ofbiz.apache.org:8080/ecommerce/control/main >> https://demo-trunk.ofbiz.apache.org/accounting/control/FindBillingAccount?partyId=DemoCustomer >> >> >> https://demo-trunk.ofbiz.apache.org/webtools/control/login >> https://demo-trunk.ofbiz.apache.org/partymgr/control/editpartygroup?partyId=DemoSupplier >> >> >> https://demo-trunk.ofbiz.apache.org/webtools/control/FindGeneric?entityName=Product&find=true&VIEW_SIZE=50&VIEW_INDEX=0 >> > > So, 5 requests issued at all once. Should be easy to duplicate that. > Start up a local ofbiz, hit then right after startup, then hit them > again after the cache(s) are primed. > >> lately when they load >> http://monitoring.apache.org/status/ >> shows everthing is ok >> I then load the above URLs and get a warning then flap then shutdown. > > That status link can't be used for debugging these kinds of problems. > It's the as saying "I found a problem in a groovy file, but the machine > pings, so I'm at a loss as to what is going on." > |
|
|
In reply to this post by Jacques Le Roux
that works for me. Count me in.
========================= BJ Freeman Strategic Power Office with Supplier Automation <http://www.businessesnetwork.com/automation/viewforum.php?f=52> Specialtymarket.com <http://www.specialtymarket.com/> Systems Integrator-- Glad to Assist Chat Y! messenger: bjfr33man Jacques Le Roux sent the following on 12/9/2010 3:42 PM: > From: "Adam Heath" <[hidden email]> >> On 12/09/2010 05:18 PM, Jacques Le Roux wrote: >>> I have spent a lot of time (I mean in respect of my free time) this last >>> days to understand the problems. >>> It appears that removing the help was a great relief for our demo sever. >>> >>> For few hours now we are running with >>> >>> trunk: -Xms128M -Xmx768M -XX:MaxPermSize=192m (so max seems >>> 768+192=960MB but actually it's more) >>> branch9: -Xms128M -Xmx512M >>> >>> For instance now we have >>> Mem: 2573924k total, 2159888k used, 414036k free, 53672k buffers >>> Swap: 1502036k total, 50676k used, 1451360k free, 438000k cached >>> >>> PID USER PR NI VIRT RES SHR %CPU %MEM >>> trunk 14896 ofbiz 20 0 1377m 753m 7956 0.3 30.0 >>> branch9 18147 ofbiz 20 0 918m 670m 13m 0.7 26.7 >>> >>> As you can see at some stage we reach more than 960MB for the trunk >>> (1377 max, which is approx but anyway) >>> >>> The main points: >>> * We have still around 400MB free, but I suppose it will be less just >>> before the 24h reload) >>> * We have anymore CPU running always near 100%, for instance right now >>> PID USER PR NI VIRT RES SHR %CPU %MEM TIME+ COMMAND >>> 14896 ofbiz 20 0 1377m 757m 7968 29.7 30.2 19:57.63 java -Xms128M >>> -Xmx768M -XX:MaxPermSize=192m >>> 18147 ofbiz 20 0 918m 671m 13m 22.4 26.7 14:23.55 java -Xms128M -Xmx512M >>> >>> >>> I will wait some days and, if things continue to go well, will re-use >>> more memory for our 2 processes. But I know there are other >>> problems... >>> >>> Like David and Scott said if people are using the Artifact Info or other >>> gluttonous features (Birts?) we will be in trouble with our >>> memory quota. So if such things come back in the future I will suggest >>> to prevent users to use them on the demo server... >>> >>> For the real problems, I think we should focus on fixing the online Help >>> feature. It seems that this isues is something relatively >>> new and a disect should help (I use this word because it's convenient, >>> on my side I simply use dichotomic tests with svn but I have >>> bigger fish to fry for now, that's why I have deactivated it). I think >>> it's not more than few days (weeks?), help appreciated... >> >> Hate to disappoint, but all those memory stats you posted are >> completely useless for actually tracking down what java is doing. >> >> You need to become friends with jmap, jhat(both standard jdk tools), >> and ibm's heap analyzer. Plus, sending the QUIT signal to the java >> process. > > Yes I know, this is only to give a general information about what's > going on on the server. > As I have already wrote I'm actually using mat > http://www.eclipse.org/mat/ behind the scene > I'm also using -XX:+HeapDumpOnOutOfMemoryError but most of the time get > rather out of swap issues when crashing, hard to trace... > One way would be mod_log_forensic... if someone wants to help... > >> In all honesty, I'm going to go out on a limb here and say the higher >> memory requirements of newer ofbiz is due to converting tons of code >> to groovy. With it as a simple-method, or a bsh, both would end up >> using heap, as they are interpetted. java or groovy get compiled to >> bytecode, which ends up being allocated in the permgen area, which >> might also get jit compiled. So, permgen needs to increase. > > It does not seem that we have permgen issues. It's not yet clear, but > for those interested I could move hprof files from demo roots to > bigfiles dir... > > Thanks > > Jacques > > > |
|
|
In reply to this post by Adam Heath-2
You got me there Adam.
and I am not saying we test or run upgrade at each update of 10.4. based on efforts so far I don't expect the effort I outlined to be done by the volunteers, though they are the best qualified to do this. as you see I do take my support of ofbiz seriously. and when you multiply this more than 50 times it turns out to be more than a man year to release what I consider stable systems. ========================= BJ Freeman Strategic Power Office with Supplier Automation <http://www.businessesnetwork.com/automation/viewforum.php?f=52> Specialtymarket.com <http://www.specialtymarket.com/> Systems Integrator-- Glad to Assist Chat Y! messenger: bjfr33man Adam Heath sent the following on 12/9/2010 3:52 PM: > On 12/09/2010 05:50 PM, BJ Freeman wrote: >> all my changes are in seperate components. >> currently my attention is 10.4 branch, which I keep updating in a test >> instance. >> 1)load and configure my componets >> 2)ant run-install >> check logs and operation related to my components. >> what usually happens is I get log errors do to changes that are not >> compatible from 9.04. >> 3)connect to postgresql >> 4)load data from production server into this test instance. >> set entityengine so will not update db. >> 5)run code so get logs reports of data changes >> 6)see if any effect my components >> 7)got through the compoents to change. >> 8)ant clean-all >> 9)clean the postresql and turn back on add tables. >> 10)run migration of production data to changed db for 10.4 >> >> first time workeffort was a few manweeks. >> each subsequence update of the 10.4 does not exceed a manweek. > > Interesting. Yes, I agree, a defined release branch in ofbiz(10.4 in > this case) should have such upgrade support from one minor point release > to another. > > However, there actually hasn't been a real release of that branch, so, ... > |
|
|
On 12/09/2010 06:09 PM, BJ Freeman wrote:
> You got me there Adam. > and I am not saying we test or run upgrade at each update of 10.4. > based on efforts so far I don't expect the effort I outlined to be done > by the volunteers, though they are the best qualified to do this. > > as you see I do take my support of ofbiz seriously. > and when you multiply this more than 50 times it turns out to be more > than a man year to release what I consider stable systems. We don't upgrade production systems. We leave them alone. If there are problems, we backport changes into whatever version of ofbiz is deployed. For the last 3 years, that has been thru a debian package of ofbiz. When a client has a problem, and it gets fixed, using the debian package allows us to get that same fix out to other production instances running the same version. We also manage the debian package thru git, and it's possible to checkout the exact version of the debian package locally onto the production system. We then change OFBIZ_HOME in the init script to point to the checkout, and can then fix the code in place. Once the fix works, it is copied into the build system, and a new package is produced. Full ofbiz package upgrades are never done without a client paying. I currently have ofbiz packages of 595296, 811564, and 902021 that I have to maintain. There are some really old versions, where we would take a snapshot of whatever ofbiz was current at the time the new job was created, but we've moved away from that. Instead, the company as a whole decided to stick with a version of ofbiz for an extended period. This period tends to be yearly(but that's not set in stone). |
«
Return to OFBiz - Dev
|
1 view|%1 views
| Free forum by Nabble | Edit this page |