Scaling ofbiz

[rajesh_r_j]

We are having severe performance issue running ofbiz with embedded tomcat. Currently it struggles to serve 50,000 pages/day (with maximum access between 7pm -11pm). Our deployment architecture is currently one linux box (2 GHz single processer, 2 GB RAM, CentOS Linux5) hosting web server (ofbiz embedded tomcat) and DB (postgresql).  

We are looking to serve 30,000 pages/day with each visit serving approximately 10 pages, which boils down to serving around 2500 - 3000 customers/day. To achieve this, we are seriously considering moving to a better server configuration and at the same time thinking about scaling up the deployment architecture (using apache for static content, having 2 – 3 instance of servers running ofbiz and having a different server for DB).

Being a new bee to ofbiz, I was reading through the forums and documentations to find out what is the correct approach to achieve the above and was surprised to see that with one box people have achieved around 1 million pages/day.

I would like to find out what will be the best approach for us to take and also find some cool links/documentation/discussions about scaling up with ofbiz(embedded tomcat), using apache for static content and separate DB.

Thanks for the help.

Raj

[BJ Freeman]

to start you evaluation:
1) watch the logs, they that the time on them so you can see how long
things take on the server you have now.
2) monitor your DB connections and watch the cpu usage for the dB
compared to ofbiz.
Use Apache httpd for static content for fastest delivery.
Next step is to move the DB to a server of its own and have a network
that supports the throughput necessary not to slow down ofbiz.
You must do DB maintianence on a regular basis.

then look at server load balancing.


rajesh_r_j sent the following on 9/1/2008 5:21 PM:

> We are having severe performance issue running ofbiz with embedded tomcat.
> Currently it struggles to serve 50,000 pages/day (with maximum access
> between 7pm -11pm). Our deployment architecture is currently one linux box
> (2 GHz single processer, 2 GB RAM, CentOS Linux5) hosting web server (ofbiz
> embedded tomcat) and DB (postgresql).  
>
> We are looking to serve 30,000 pages/day with each visit serving
> approximately 10 pages, which boils down to serving around 2500 - 3000
> customers/day. To achieve this, we are seriously considering moving to a
> better server configuration and at the same time thinking about scaling up
> the deployment architecture (using apache for static content, having 2 – 3
> instance of servers running ofbiz and having a different server for DB).
>
> Being a new bee to ofbiz, I was reading through the forums and
> documentations to find out what is the correct approach to achieve the above
> and was surprised to see that with one box people have achieved around 1
> million pages/day.
>
> I would like to find out what will be the best approach for us to take and
> also find some cool links/documentation/discussions about scaling up with
> ofbiz(embedded tomcat), using apache for static content and separate DB.
>
> Thanks for the help.
>
> Raj

[BJ Freeman]

there is a link
http://docs.ofbiz.org/display/OFBTECH/Configuration,+Deployment+and+Maintenance

BJ Freeman sent the following on 9/1/2008 5:44 PM:

> to start you evaluation:
> 1) watch the logs, they that the time on them so you can see how long
> things take on the server you have now.
> 2) monitor your DB connections and watch the cpu usage for the dB
> compared to ofbiz.
> Use Apache httpd for static content for fastest delivery.
> Next step is to move the DB to a server of its own and have a network
> that supports the throughput necessary not to slow down ofbiz.
> You must do DB maintianence on a regular basis.
>
> then look at server load balancing.
>
>
> rajesh_r_j sent the following on 9/1/2008 5:21 PM:
>> We are having severe performance issue running ofbiz with embedded tomcat.
>> Currently it struggles to serve 50,000 pages/day (with maximum access
>> between 7pm -11pm). Our deployment architecture is currently one linux box
>> (2 GHz single processer, 2 GB RAM, CentOS Linux5) hosting web server (ofbiz
>> embedded tomcat) and DB (postgresql).  
>>
>> We are looking to serve 30,000 pages/day with each visit serving
>> approximately 10 pages, which boils down to serving around 2500 - 3000
>> customers/day. To achieve this, we are seriously considering moving to a
>> better server configuration and at the same time thinking about scaling up
>> the deployment architecture (using apache for static content, having 2 – 3
>> instance of servers running ofbiz and having a different server for DB).
>>
>> Being a new bee to ofbiz, I was reading through the forums and
>> documentations to find out what is the correct approach to achieve the above
>> and was surprised to see that with one box people have achieved around 1
>> million pages/day.
>>
>> I would like to find out what will be the best approach for us to take and
>> also find some cool links/documentation/discussions about scaling up with
>> ofbiz(embedded tomcat), using apache for static content and separate DB.
>>
>> Thanks for the help.
>>
>> Raj
>
>
>
>
>

[madppiper-2]

I just skipped through the Database and noticed how it bursts with Varchar, Longtext and Decimal Datatypes. With increasing Datasets that should result in slower query-results and hence use up some of the needed CPU-Performance.

(The reason for that is similar to the following:
If you think of the database as of a huge table that the program has to skip through, it is best if each cell has a specific length. That way you can skip through the end of each cell with ease.
Now with Varchars and Longtext datatypes you cannot do any of that. The Datatypes are variable in length (even if you set the maximum length - as done in the database) and hence slow down the process, since you have to read through the entire cell before noticing the end... )

As such, i would argue for a revision of the database-structure. I haven't taken a deeper look at the table Indexes (those can largely affect the overall performance as well), but i think that we can easily increase the system performance by turning varchars into chars with maximum lengths...

I know that that also means an increased need for diskpace (the maximum length of a cell is always reserved), but in my opinion, diskspace comes much cheaper nowadays than a heated need for high performin cpus...

rajesh_r_j wrote

We are having severe performance issue running ofbiz with embedded tomcat. Currently it struggles to serve 50,000 pages/day (with maximum access between 7pm -11pm). Our deployment architecture is currently one linux box (2 GHz single processer, 2 GB RAM, CentOS Linux5) hosting web server (ofbiz embedded tomcat) and DB (postgresql).  

We are looking to serve 30,000 pages/day with each visit serving approximately 10 pages, which boils down to serving around 2500 - 3000 customers/day. To achieve this, we are seriously considering moving to a better server configuration and at the same time thinking about scaling up the deployment architecture (using apache for static content, having 2 – 3 instance of servers running ofbiz and having a different server for DB).

Being a new bee to ofbiz, I was reading through the forums and documentations to find out what is the correct approach to achieve the above and was surprised to see that with one box people have achieved around 1 million pages/day.

I would like to find out what will be the best approach for us to take and also find some cool links/documentation/discussions about scaling up with ofbiz(embedded tomcat), using apache for static content and separate DB.

Thanks for the help.

Raj

[BJ Freeman]

this should be on the User Mailing list.
I have copied this to the User Mailing list.
there is a response from David to this with the subject
Re: Deploy Ofbiz in JBoss

madppiper sent the following on 9/2/2008 1:45 AM:

> I just skipped through the Database and noticed how it bursts with Varchar,
> Longtext and Decimal Datatypes. With increasing Datasets that should result
> in slower query-results and hence use up some of the needed CPU-Performance.
>
> (The reason for that is similar to the following:
> If you think of the database as of a huge table that the program has to skip
> through, it is best if each cell has a specific length. That way you can
> skip through the end of each cell with ease.
> Now with Varchars and Longtext datatypes you cannot do any of that. The
> Datatypes are variable in length (even if you set the maximum length - as
> done in the database) and hence slow down the process, since you have to
> read through the entire cell before noticing the end... )
>
> As such, i would argue for a revision of the database-structure. I haven't
> taken a deeper look at the table Indexes (those can largely affect the
> overall performance as well), but i think that we can easily increase the
> system performance by turning varchars into chars with maximum lengths...
>
> I know that that also means an increased need for diskpace (the maximum
> length of a cell is always reserved), but in my opinion, diskspace comes
> much cheaper nowadays than a heated need for high performin cpus...
>
>
>
> rajesh_r_j wrote:
>> We are having severe performance issue running ofbiz with embedded tomcat.
>> Currently it struggles to serve 50,000 pages/day (with maximum access
>> between 7pm -11pm). Our deployment architecture is currently one linux box
>> (2 GHz single processer, 2 GB RAM, CentOS Linux5) hosting web server
>> (ofbiz embedded tomcat) and DB (postgresql).  
>>
>> We are looking to serve 30,000 pages/day with each visit serving
>> approximately 10 pages, which boils down to serving around 2500 - 3000
>> customers/day. To achieve this, we are seriously considering moving to a
>> better server configuration and at the same time thinking about scaling up
>> the deployment architecture (using apache for static content, having 2 – 3
>> instance of servers running ofbiz and having a different server for DB).
>>
>> Being a new bee to ofbiz, I was reading through the forums and
>> documentations to find out what is the correct approach to achieve the
>> above and was surprised to see that with one box people have achieved
>> around 1 million pages/day.
>>
>> I would like to find out what will be the best approach for us to take and
>> also find some cool links/documentation/discussions about scaling up with
>> ofbiz(embedded tomcat), using apache for static content and separate DB.
>>
>> Thanks for the help.
>>
>> Raj
>>
>

[rajesh_r_j]

The link just contains the index of document not document itself.  

For the current setup i followed Technical Production Setup Guide (http://docs.ofbiz.org/display/OFBTECH/Apache+OFBiz+Technical+Production+Setup+Guide) but do not get the performance anywhere near what others have claimed.

I also had a look at the ofbiz logs, cpu timing and memory usage for ofbiz and postgresql process and did not see anything unusual, but everything was just slow and at this rate cannot serve 30,000 pages/day (95% of the visits during 7pm to 11pm).

May be it is the hardware, does anyone have a performance matrix based on hardware configuration or can someone suggest the hardware configuration i need to have to support 10,000 page visits/hour.

Also it will be good if someone can point me toward steps or documentation that explains how to scale/setup ofbiz to run on multiple servers(load balancing & clustering) and also how to front ofbiz with apache server for static content.

thanks
Raj

BJ Freeman wrote

there is a link
http://docs.ofbiz.org/display/OFBTECH/Configuration,+Deployment+and+Maintenance

BJ Freeman sent the following on 9/1/2008 5:44 PM:
> to start you evaluation:
> 1) watch the logs, they that the time on them so you can see how long
> things take on the server you have now.
> 2) monitor your DB connections and watch the cpu usage for the dB
> compared to ofbiz.
> Use Apache httpd for static content for fastest delivery.
> Next step is to move the DB to a server of its own and have a network
> that supports the throughput necessary not to slow down ofbiz.
> You must do DB maintianence on a regular basis.
>
> then look at server load balancing.
>
>
> rajesh_r_j sent the following on 9/1/2008 5:21 PM:
>> We are having severe performance issue running ofbiz with embedded tomcat.
>> Currently it struggles to serve 50,000 pages/day (with maximum access
>> between 7pm -11pm). Our deployment architecture is currently one linux box
>> (2 GHz single processer, 2 GB RAM, CentOS Linux5) hosting web server (ofbiz
>> embedded tomcat) and DB (postgresql).  
>>
>> We are looking to serve 30,000 pages/day with each visit serving
>> approximately 10 pages, which boils down to serving around 2500 - 3000
>> customers/day. To achieve this, we are seriously considering moving to a
>> better server configuration and at the same time thinking about scaling up
>> the deployment architecture (using apache for static content, having 2 – 3
>> instance of servers running ofbiz and having a different server for DB).
>>
>> Being a new bee to ofbiz, I was reading through the forums and
>> documentations to find out what is the correct approach to achieve the above
>> and was surprised to see that with one box people have achieved around 1
>> million pages/day.
>>
>> I would like to find out what will be the best approach for us to take and
>> also find some cool links/documentation/discussions about scaling up with
>> ofbiz(embedded tomcat), using apache for static content and separate DB.
>>
>> Thanks for the help.
>>
>> Raj
>
>
>
>
>