VISIT and VISITOR Entities
Locked
 
|
All,
We have a LIVE application in which the VISIT AND VISITOR entities have grown to over 19 and 18 million rows respectively. Has anyone experienced any performance issues when these entities have grown to such capacity? We recently had a performance issue and decided to turn off the visit history feature until this feature can be ruled out as the issue. stats.persist.visit=false stats.persist.visitor=false Note: The application is currently using Ofbiz r.10. Len Shein <mailto:[hidden email]> [hidden email] O: 516.742.7888 x225 O: 732.333.4303 C: 917.882.8515 |
|
|
Did the setting change improve performance?
You could write a service that deletes data older than xxx days, then make it a scheduled job. Adrian Crum Sandglass Software www.sandglass-software.com On 11/24/2015 7:48 AM, Len Shein wrote: > All, > > We have a LIVE application in which the VISIT AND VISITOR entities have > grown to over 19 and 18 million rows respectively. > > Has anyone experienced any performance issues when these entities have grown > to such capacity? > > > > We recently had a performance issue and decided to turn off the visit > history feature until this feature can be ruled out as the issue. > > stats.persist.visit=false > > stats.persist.visitor=false > > > > Note: The application is currently using Ofbiz r.10. > > > Len Shein > > <mailto:[hidden email]> [hidden email] > > O: 516.742.7888 x225 > > O: 732.333.4303 > C: 917.882.8515 > > |
|
|
Adrian,
Not entirely sure if it made a difference -- site was going down every now and again, initially we thought that some scheduled jobs were the culprit (since outages were in or around the scheduled job time). However, we had problems yesterday when no scheduled jobs were running (we disabled most of them for now). Site has been fine since we turned off VISIT and VISITOR tracking -- can't be 100% sure that this is the resolution factor given general problems over the last couple of weeks. I guess that if the site is fine for several days then we would have to attribute it to the serverstats change. We've heard of problems when the JOB_SANDBOX gets very large -- ever heard about instability when VISIT and VISITORS get very large? (19M and 18M rows each). Nick -----Original Message----- From: Adrian Crum [mailto:[hidden email]] Sent: Tuesday, November 24, 2015 11:13 AM To: [hidden email] Subject: Re: VISIT and VISITOR Entities Did the setting change improve performance? You could write a service that deletes data older than xxx days, then make it a scheduled job. Adrian Crum Sandglass Software www.sandglass-software.com On 11/24/2015 7:48 AM, Len Shein wrote: > All, > > We have a LIVE application in which the VISIT AND VISITOR entities > have grown to over 19 and 18 million rows respectively. > > Has anyone experienced any performance issues when these entities have > grown to such capacity? > > > > We recently had a performance issue and decided to turn off the visit > history feature until this feature can be ruled out as the issue. > > stats.persist.visit=false > > stats.persist.visitor=false > > > > Note: The application is currently using Ofbiz r.10. > > > Len Shein > > <mailto:[hidden email]> [hidden email] > > O: 516.742.7888 x225 > > O: 732.333.4303 > C: 917.882.8515 > > |
|
|
In reply to this post by Len Shein-3
Most likely this table is stealing cache memory from the rest of the
tables. Either increase cache memory, delete the rows, write to a separate database, or keep the table trim to xxx rows (as Adrian said) via an internal ofbiz job or external script (via crontab). On Tue, Nov 24, 2015 at 7:48 AM, Len Shein <[hidden email]> wrote: > All, > > We have a LIVE application in which the VISIT AND VISITOR entities have > grown to over 19 and 18 million rows respectively. > > Has anyone experienced any performance issues when these entities have > grown > to such capacity? > > > > We recently had a performance issue and decided to turn off the visit > history feature until this feature can be ruled out as the issue. > > stats.persist.visit=false > > stats.persist.visitor=false > > > > Note: The application is currently using Ofbiz r.10. > > > Len Shein > > <mailto:[hidden email]> [hidden email] > > O: 516.742.7888 x225 > > O: 732.333.4303 > C: 917.882.8515 > > |
|
|
Thanks Mike -- don't think this is a cache issue -- at least not OFBiz cache since it is not showing up in the OFBiz Cache maintenance page.
Nick -----Original Message----- From: Mike [mailto:[hidden email]] Sent: Tuesday, November 24, 2015 11:31 AM To: user <[hidden email]> Subject: Re: VISIT and VISITOR Entities Most likely this table is stealing cache memory from the rest of the tables. Either increase cache memory, delete the rows, write to a separate database, or keep the table trim to xxx rows (as Adrian said) via an internal ofbiz job or external script (via crontab). On Tue, Nov 24, 2015 at 7:48 AM, Len Shein <[hidden email]> wrote: > All, > > We have a LIVE application in which the VISIT AND VISITOR entities > have grown to over 19 and 18 million rows respectively. > > Has anyone experienced any performance issues when these entities have > grown to such capacity? > > > > We recently had a performance issue and decided to turn off the visit > history feature until this feature can be ruled out as the issue. > > stats.persist.visit=false > > stats.persist.visitor=false > > > > Note: The application is currently using Ofbiz r.10. > > > Len Shein > > <mailto:[hidden email]> [hidden email] > > O: 516.742.7888 x225 > > O: 732.333.4303 > C: 917.882.8515 > > |
|
|
In reply to this post by Nick Rosser-2
Hmmm... it could be the job scheduler. It was refactored a few years ago
to fix problems like that. Try C&Ping R13 job scheduler into R10 (framework\service\src\org\ofbiz\service\job). Adrian Crum Sandglass Software www.sandglass-software.com On 11/24/2015 8:29 AM, Nick Rosser wrote: > Adrian, > > Not entirely sure if it made a difference -- site was going down every now and again, initially we thought that some scheduled jobs were the culprit (since outages were in or around the scheduled job time). However, we had problems yesterday when no scheduled jobs were running (we disabled most of them for now). > > Site has been fine since we turned off VISIT and VISITOR tracking -- can't be 100% sure that this is the resolution factor given general problems over the last couple of weeks. I guess that if the site is fine for several days then we would have to attribute it to the serverstats change. > > We've heard of problems when the JOB_SANDBOX gets very large -- ever heard about instability when VISIT and VISITORS get very large? (19M and 18M rows each). > > Nick > > -----Original Message----- > From: Adrian Crum [mailto:[hidden email]] > Sent: Tuesday, November 24, 2015 11:13 AM > To: [hidden email] > Subject: Re: VISIT and VISITOR Entities > > Did the setting change improve performance? > > You could write a service that deletes data older than xxx days, then make it a scheduled job. > > Adrian Crum > Sandglass Software > www.sandglass-software.com > > On 11/24/2015 7:48 AM, Len Shein wrote: >> All, >> >> We have a LIVE application in which the VISIT AND VISITOR entities >> have grown to over 19 and 18 million rows respectively. >> >> Has anyone experienced any performance issues when these entities have >> grown to such capacity? >> >> >> >> We recently had a performance issue and decided to turn off the visit >> history feature until this feature can be ruled out as the issue. >> >> stats.persist.visit=false >> >> stats.persist.visitor=false >> >> >> >> Note: The application is currently using Ofbiz r.10. >> >> >> Len Shein >> >> <mailto:[hidden email]> [hidden email] >> >> O: 516.742.7888 x225 >> >> O: 732.333.4303 >> C: 917.882.8515 >> >> |
|
|
Thanks Adrian -- we'll take a look at the latest version.
Maybe a complete red herring -- but a couple of weeks ago our client requested that we change the password for the primary "admin" user. We deleted and re-added main scheduled jobs but we've had problems since ... Nick -----Original Message----- From: Adrian Crum [mailto:[hidden email]] Sent: Tuesday, November 24, 2015 11:58 AM To: [hidden email] Subject: Re: VISIT and VISITOR Entities Hmmm... it could be the job scheduler. It was refactored a few years ago to fix problems like that. Try C&Ping R13 job scheduler into R10 (framework\service\src\org\ofbiz\service\job). Adrian Crum Sandglass Software www.sandglass-software.com On 11/24/2015 8:29 AM, Nick Rosser wrote: > Adrian, > > Not entirely sure if it made a difference -- site was going down every now and again, initially we thought that some scheduled jobs were the culprit (since outages were in or around the scheduled job time). However, we had problems yesterday when no scheduled jobs were running (we disabled most of them for now). > > Site has been fine since we turned off VISIT and VISITOR tracking -- can't be 100% sure that this is the resolution factor given general problems over the last couple of weeks. I guess that if the site is fine for several days then we would have to attribute it to the serverstats change. > > We've heard of problems when the JOB_SANDBOX gets very large -- ever heard about instability when VISIT and VISITORS get very large? (19M and 18M rows each). > > Nick > > -----Original Message----- > From: Adrian Crum [mailto:[hidden email]] > Sent: Tuesday, November 24, 2015 11:13 AM > To: [hidden email] > Subject: Re: VISIT and VISITOR Entities > > Did the setting change improve performance? > > You could write a service that deletes data older than xxx days, then make it a scheduled job. > > Adrian Crum > Sandglass Software > www.sandglass-software.com > > On 11/24/2015 7:48 AM, Len Shein wrote: >> All, >> >> We have a LIVE application in which the VISIT AND VISITOR entities >> have grown to over 19 and 18 million rows respectively. >> >> Has anyone experienced any performance issues when these entities >> have grown to such capacity? >> >> >> >> We recently had a performance issue and decided to turn off the visit >> history feature until this feature can be ruled out as the issue. >> >> stats.persist.visit=false >> >> stats.persist.visitor=false >> >> >> >> Note: The application is currently using Ofbiz r.10. >> >> >> Len Shein >> >> <mailto:[hidden email]> [hidden email] >> >> O: 516.742.7888 x225 >> >> O: 732.333.4303 >> C: 917.882.8515 >> >> |
|
|
In reply to this post by Nick Rosser-2
> at least not OFBiz cache since
I didn't mean the ofbiz cache. I meant the database (mysql/postgres) cache settings. On Tue, Nov 24, 2015 at 8:34 AM, Nick Rosser <[hidden email]> wrote: > Thanks Mike -- don't think this is a cache issue -- at least not OFBiz > cache since it is not showing up in the OFBiz Cache maintenance page. > Nick > > -----Original Message----- > From: Mike [mailto:[hidden email]] > Sent: Tuesday, November 24, 2015 11:31 AM > To: user <[hidden email]> > Subject: Re: VISIT and VISITOR Entities > > Most likely this table is stealing cache memory from the rest of the > tables. Either increase cache memory, delete the rows, write to a separate > database, or keep the table trim to xxx rows (as Adrian said) via an > internal ofbiz job or external script (via crontab). > > On Tue, Nov 24, 2015 at 7:48 AM, Len Shein <[hidden email]> wrote: > > > All, > > > > We have a LIVE application in which the VISIT AND VISITOR entities > > have grown to over 19 and 18 million rows respectively. > > > > Has anyone experienced any performance issues when these entities have > > grown to such capacity? > > > > > > > > We recently had a performance issue and decided to turn off the visit > > history feature until this feature can be ruled out as the issue. > > > > stats.persist.visit=false > > > > stats.persist.visitor=false > > > > > > > > Note: The application is currently using Ofbiz r.10. > > > > > > Len Shein > > > > <mailto:[hidden email]> [hidden email] > > > > O: 516.742.7888 x225 > > > > O: 732.333.4303 > > C: 917.882.8515 > > > > > |
|
|
Got it, thanks Mike.
Nick -----Original Message----- From: Mike [mailto:[hidden email]] Sent: Tuesday, November 24, 2015 12:23 PM To: user <[hidden email]> Subject: Re: VISIT and VISITOR Entities > at least not OFBiz cache since I didn't mean the ofbiz cache. I meant the database (mysql/postgres) cache settings. On Tue, Nov 24, 2015 at 8:34 AM, Nick Rosser <[hidden email]> wrote: > Thanks Mike -- don't think this is a cache issue -- at least not OFBiz > cache since it is not showing up in the OFBiz Cache maintenance page. > Nick > > -----Original Message----- > From: Mike [mailto:[hidden email]] > Sent: Tuesday, November 24, 2015 11:31 AM > To: user <[hidden email]> > Subject: Re: VISIT and VISITOR Entities > > Most likely this table is stealing cache memory from the rest of the > tables. Either increase cache memory, delete the rows, write to a > separate database, or keep the table trim to xxx rows (as Adrian said) > via an internal ofbiz job or external script (via crontab). > > On Tue, Nov 24, 2015 at 7:48 AM, Len Shein <[hidden email]> wrote: > > > All, > > > > We have a LIVE application in which the VISIT AND VISITOR entities > > have grown to over 19 and 18 million rows respectively. > > > > Has anyone experienced any performance issues when these entities > > have grown to such capacity? > > > > > > > > We recently had a performance issue and decided to turn off the > > visit history feature until this feature can be ruled out as the issue. > > > > stats.persist.visit=false > > > > stats.persist.visitor=false > > > > > > > > Note: The application is currently using Ofbiz r.10. > > > > > > Len Shein > > > > <mailto:[hidden email]> [hidden email] > > > > O: 516.742.7888 x225 > > > > O: 732.333.4303 > > C: 917.882.8515 > > > > > |
Re: VISIT and VISITOR Entities
Administrator
|
That's why using another DB, as suggested Mike ("write to a separate database"), could be a good test...
Jacques Le 24/11/2015 18:23, Nick Rosser a écrit : > Got it, thanks Mike. > Nick > > -----Original Message----- > From: Mike [mailto:[hidden email]] > Sent: Tuesday, November 24, 2015 12:23 PM > To: user <[hidden email]> > Subject: Re: VISIT and VISITOR Entities > >> at least not OFBiz cache since > I didn't mean the ofbiz cache. I meant the database (mysql/postgres) cache settings. > > On Tue, Nov 24, 2015 at 8:34 AM, Nick Rosser <[hidden email]> wrote: > >> Thanks Mike -- don't think this is a cache issue -- at least not OFBiz >> cache since it is not showing up in the OFBiz Cache maintenance page. >> Nick >> >> -----Original Message----- >> From: Mike [mailto:[hidden email]] >> Sent: Tuesday, November 24, 2015 11:31 AM >> To: user <[hidden email]> >> Subject: Re: VISIT and VISITOR Entities >> >> Most likely this table is stealing cache memory from the rest of the >> tables. Either increase cache memory, delete the rows, write to a >> separate database, or keep the table trim to xxx rows (as Adrian said) >> via an internal ofbiz job or external script (via crontab). >> >> On Tue, Nov 24, 2015 at 7:48 AM, Len Shein <[hidden email]> wrote: >> >>> All, >>> >>> We have a LIVE application in which the VISIT AND VISITOR entities >>> have grown to over 19 and 18 million rows respectively. >>> >>> Has anyone experienced any performance issues when these entities >>> have grown to such capacity? >>> >>> >>> >>> We recently had a performance issue and decided to turn off the >>> visit history feature until this feature can be ruled out as the issue. >>> >>> stats.persist.visit=false >>> >>> stats.persist.visitor=false >>> >>> >>> >>> Note: The application is currently using Ofbiz r.10. >>> >>> >>> Len Shein >>> >>> <mailto:[hidden email]> [hidden email] >>> >>> O: 516.742.7888 x225 >>> >>> O: 732.333.4303 >>> C: 917.882.8515 >>> >>> > |
|
|
In reply to this post by Mike Z
For mysql, there is tuning_primer.sh:
https://launchpad.net/mysql-tuning-primer For postgresql, there is pgtune: http://pgfoundry.org/projects/pgtune Most folks just use yum/apt-get to get pgtune: apt-get install pgtune --or-- yum install pgtune Although neither are perfect, both utilities attempt to change the DB settings to take advantage of hardware, or look at certain things, like the number of tables, and adjust the DB settings for maximum performance. The default settings for mysql are quite limiting for ofbiz. For instance, ofbiz has over 800 tables and the default table cache for mysql is like 60 tables. On Tue, Nov 24, 2015 at 9:22 AM, Mike <[hidden email]> wrote: > > at least not OFBiz cache since > > I didn't mean the ofbiz cache. I meant the database (mysql/postgres) > cache settings. > > On Tue, Nov 24, 2015 at 8:34 AM, Nick Rosser <[hidden email]> wrote: > >> Thanks Mike -- don't think this is a cache issue -- at least not OFBiz >> cache since it is not showing up in the OFBiz Cache maintenance page. >> Nick >> >> -----Original Message----- >> From: Mike [mailto:[hidden email]] >> Sent: Tuesday, November 24, 2015 11:31 AM >> To: user <[hidden email]> >> Subject: Re: VISIT and VISITOR Entities >> >> Most likely this table is stealing cache memory from the rest of the >> tables. Either increase cache memory, delete the rows, write to a separate >> database, or keep the table trim to xxx rows (as Adrian said) via an >> internal ofbiz job or external script (via crontab). >> >> On Tue, Nov 24, 2015 at 7:48 AM, Len Shein <[hidden email]> wrote: >> >> > All, >> > >> > We have a LIVE application in which the VISIT AND VISITOR entities >> > have grown to over 19 and 18 million rows respectively. >> > >> > Has anyone experienced any performance issues when these entities have >> > grown to such capacity? >> > >> > >> > >> > We recently had a performance issue and decided to turn off the visit >> > history feature until this feature can be ruled out as the issue. >> > >> > stats.persist.visit=false >> > >> > stats.persist.visitor=false >> > >> > >> > >> > Note: The application is currently using Ofbiz r.10. >> > >> > >> > Len Shein >> > >> > <mailto:[hidden email]> [hidden email] >> > >> > O: 516.742.7888 x225 >> > >> > O: 732.333.4303 >> > C: 917.882.8515 >> > >> > >> > > |
«
Return to OFBiz - User
|
1 view|%1 views
| Free forum by Nabble | Edit this page |