OFBiz Caching and Memory Documentation

[Nick Rosser]

This is a guide that we put together as we were coming to terms with the
OFBiz caching and memory scheme. Existing documentation, as far as we
could tell, was just about non-existent. Since the caching scheme is so
important from a performance and memory usage point of view we decided
to document our findings.

For anyone's review -- we'd be interested in any feedback:

http://bigfish.salmonllc.com/help/guideCachingAndMemory.htm

--
Best Regards,

Nick Rosser
[hidden email]
O: 516.742.7888 x221
C: 516.901.1720

[Adrian Crum-3]

"Use File Store? True indicates that the cache can grow beyond the
memory limit and will push additional elements to disk. This has limited
usefulness since the processing required to integrate with the file
system is roughly equivalent to retrieving from a database. We recommend
that this is set to false unless there is specific knowledge or
requirements to use a File Store."

That statement is overly simplistic. It assumes caches are used solely
for storing database data - when they are also used to store OFBiz
internal structures. Caching to disk can reduce the time required to
retrieve/parse many things that are not stored in the database.

-Adrian

On 12/18/2012 3:39 PM, Nick Rosser wrote:

> This is a guide that we put together as we were coming to terms with
> the OFBiz caching and memory scheme. Existing documentation, as far as
> we could tell, was just about non-existent. Since the caching scheme
> is so important from a performance and memory usage point of view we
> decided to document our findings.
>
> For anyone's review -- we'd be interested in any feedback:
>
> http://bigfish.salmonllc.com/help/guideCachingAndMemory.htm
>

[Nick Rosser]

Adrian,

Thanks for the clarification ... point taken ... we'll update the doc
with your comment.

Would you agree that it has minimal usage when it comes to the database
caching aspect of OFBiz?

Best Regards,

Nick Rosser
[hidden email]
O: 516.742.7888 x221
C: 516.901.1720


On 12/18/2012 2:58 PM, Adrian Crum wrote:

> "Use File Store? True indicates that the cache can grow beyond the
> memory limit and will push additional elements to disk. This has
> limited usefulness since the processing required to integrate with the
> file system is roughly equivalent to retrieving from a database. We
> recommend that this is set to false unless there is specific knowledge
> or requirements to use a File Store."
>
> That statement is overly simplistic. It assumes caches are used solely
> for storing database data - when they are also used to store OFBiz
> internal structures. Caching to disk can reduce the time required to
> retrieve/parse many things that are not stored in the database.
>
> -Adrian
>
> On 12/18/2012 3:39 PM, Nick Rosser wrote:
>> This is a guide that we put together as we were coming to terms with
>> the OFBiz caching and memory scheme. Existing documentation, as far
>> as we could tell, was just about non-existent. Since the caching
>> scheme is so important from a performance and memory usage point of
>> view we decided to document our findings.
>>
>> For anyone's review -- we'd be interested in any feedback:
>>
>> http://bigfish.salmonllc.com/help/guideCachingAndMemory.htm
>>
>

[Jacques Le Roux]

I read

<<Other caches:
  a.. FTL
  b.. Groovy
  c.. XML files>>

There are also :
properties
security (minor)
workflow (deprecated, I will move this to Attic)
product.content
minilang
bsh  (deprecated)
service (model)
BsfEvents (deprecated)
JasperReports  (deprecated OOTB)
controller
widgets
datafile

See cache.properties

Jacques

From: "Nick Rosser" <[hidden email]>

> Adrian,
>
> Thanks for the clarification ... point taken ... we'll update the doc
> with your comment.
>
> Would you agree that it has minimal usage when it comes to the database
> caching aspect of OFBiz?
>
> Best Regards,
>
> Nick Rosser
> [hidden email]
> O: 516.742.7888 x221
> C: 516.901.1720
>
>
> On 12/18/2012 2:58 PM, Adrian Crum wrote:
>> "Use File Store? True indicates that the cache can grow beyond the
>> memory limit and will push additional elements to disk. This has
>> limited usefulness since the processing required to integrate with the
>> file system is roughly equivalent to retrieving from a database. We
>> recommend that this is set to false unless there is specific knowledge
>> or requirements to use a File Store."
>>
>> That statement is overly simplistic. It assumes caches are used solely
>> for storing database data - when they are also used to store OFBiz
>> internal structures. Caching to disk can reduce the time required to
>> retrieve/parse many things that are not stored in the database.
>>
>> -Adrian
>>
>> On 12/18/2012 3:39 PM, Nick Rosser wrote:
>>> This is a guide that we put together as we were coming to terms with
>>> the OFBiz caching and memory scheme. Existing documentation, as far
>>> as we could tell, was just about non-existent. Since the caching
>>> scheme is so important from a performance and memory usage point of
>>> view we decided to document our findings.
>>>
>>> For anyone's review -- we'd be interested in any feedback:
>>>
>>> http://bigfish.salmonllc.com/help/guideCachingAndMemory.htm
>>>
>>

[Nick Rosser]

OK, thanks, we'll make the change.

Is this documentation generally useful? We struggled finding anything on
the details and nuances of the whole caching scheme. And it's critical
for scalable implementations.

Best Regards,

Nick Rosser
[hidden email]
O: 516.742.7888 x221
C: 516.901.1720


On 12/19/2012 12:34 AM, Jacques Le Roux wrote:

> I read
>
> <<Other caches:
>    a.. FTL
>    b.. Groovy
>    c.. XML files>>
>
> There are also :
> properties
> security (minor)
> workflow (deprecated, I will move this to Attic)
> product.content
> minilang
> bsh  (deprecated)
> service (model)
> BsfEvents (deprecated)
> JasperReports  (deprecated OOTB)
> controller
> widgets
> datafile
>
> See cache.properties
>
> Jacques
>
> From: "Nick Rosser"<[hidden email]>
>> Adrian,
>>
>> Thanks for the clarification ... point taken ... we'll update the doc
>> with your comment.
>>
>> Would you agree that it has minimal usage when it comes to the database
>> caching aspect of OFBiz?
>>
>> Best Regards,
>>
>> Nick Rosser
>> [hidden email]
>> O: 516.742.7888 x221
>> C: 516.901.1720
>>
>>
>> On 12/18/2012 2:58 PM, Adrian Crum wrote:
>>> "Use File Store? True indicates that the cache can grow beyond the
>>> memory limit and will push additional elements to disk. This has
>>> limited usefulness since the processing required to integrate with the
>>> file system is roughly equivalent to retrieving from a database. We
>>> recommend that this is set to false unless there is specific knowledge
>>> or requirements to use a File Store."
>>>
>>> That statement is overly simplistic. It assumes caches are used solely
>>> for storing database data - when they are also used to store OFBiz
>>> internal structures. Caching to disk can reduce the time required to
>>> retrieve/parse many things that are not stored in the database.
>>>
>>> -Adrian
>>>
>>> On 12/18/2012 3:39 PM, Nick Rosser wrote:
>>>> This is a guide that we put together as we were coming to terms with
>>>> the OFBiz caching and memory scheme. Existing documentation, as far
>>>> as we could tell, was just about non-existent. Since the caching
>>>> scheme is so important from a performance and memory usage point of
>>>> view we decided to document our findings.
>>>>
>>>> For anyone's review -- we'd be interested in any feedback:
>>>>
>>>> http://bigfish.salmonllc.com/help/guideCachingAndMemory.htm
>>>>

[Mike Z]

Looks like a very useful document, which builds confidence in deploying
BigFish.  Thanks.

On Wed, Dec 19, 2012 at 5:35 AM, Nick Rosser <[hidden email]> wrote:

> OK, thanks, we'll make the change.
>
> Is this documentation generally useful? We struggled finding anything on
> the details and nuances of the whole caching scheme. And it's critical for
> scalable implementations.
>
>
> Best Regards,
>
> Nick Rosser
> [hidden email]
> O: 516.742.7888 x221
> C: 516.901.1720
>
>
> On 12/19/2012 12:34 AM, Jacques Le Roux wrote:
>
>> I read
>>
>> <<Other caches:
>>    a.. FTL
>>    b.. Groovy
>>    c.. XML files>>
>>
>> There are also :
>> properties
>> security (minor)
>> workflow (deprecated, I will move this to Attic)
>> product.content
>> minilang
>> bsh  (deprecated)
>> service (model)
>> BsfEvents (deprecated)
>> JasperReports  (deprecated OOTB)
>> controller
>> widgets
>> datafile
>>
>> See cache.properties
>>
>> Jacques
>>
>> From: "Nick Rosser"<[hidden email]>
>>
>>> Adrian,
>>>
>>> Thanks for the clarification ... point taken ... we'll update the doc
>>> with your comment.
>>>
>>> Would you agree that it has minimal usage when it comes to the database
>>> caching aspect of OFBiz?
>>>
>>> Best Regards,
>>>
>>> Nick Rosser
>>> [hidden email]
>>> O: 516.742.7888 x221
>>> C: 516.901.1720
>>>
>>>
>>> On 12/18/2012 2:58 PM, Adrian Crum wrote:
>>>
>>>> "Use File Store? True indicates that the cache can grow beyond the
>>>> memory limit and will push additional elements to disk. This has
>>>> limited usefulness since the processing required to integrate with the
>>>> file system is roughly equivalent to retrieving from a database. We
>>>> recommend that this is set to false unless there is specific knowledge
>>>> or requirements to use a File Store."
>>>>
>>>> That statement is overly simplistic. It assumes caches are used solely
>>>> for storing database data - when they are also used to store OFBiz
>>>> internal structures. Caching to disk can reduce the time required to
>>>> retrieve/parse many things that are not stored in the database.
>>>>
>>>> -Adrian
>>>>
>>>> On 12/18/2012 3:39 PM, Nick Rosser wrote:
>>>>
>>>>> This is a guide that we put together as we were coming to terms with
>>>>> the OFBiz caching and memory scheme. Existing documentation, as far
>>>>> as we could tell, was just about non-existent. Since the caching
>>>>> scheme is so important from a performance and memory usage point of
>>>>> view we decided to document our findings.
>>>>>
>>>>> For anyone's review -- we'd be interested in any feedback:
>>>>>
>>>>> http://bigfish.salmonllc.com/**help/guideCachingAndMemory.htm<http://bigfish.salmonllc.com/help/guideCachingAndMemory.htm>
>>>>>
>>>>>

[Adrian Crum-3]

Btw, caching Jasper Reports to disk will fail. Those should be set to false.

-Adrian

On 12/19/2012 5:34 AM, Jacques Le Roux wrote:

> I read
>
> <<Other caches:
>    a.. FTL
>    b.. Groovy
>    c.. XML files>>
>
> There are also :
> properties
> security (minor)
> workflow (deprecated, I will move this to Attic)
> product.content
> minilang
> bsh  (deprecated)
> service (model)
> BsfEvents (deprecated)
> JasperReports  (deprecated OOTB)
> controller
> widgets
> datafile
>
> See cache.properties
>
> Jacques
>
> From: "Nick Rosser" <[hidden email]>
>> Adrian,
>>
>> Thanks for the clarification ... point taken ... we'll update the doc
>> with your comment.
>>
>> Would you agree that it has minimal usage when it comes to the database
>> caching aspect of OFBiz?
>>
>> Best Regards,
>>
>> Nick Rosser
>> [hidden email]
>> O: 516.742.7888 x221
>> C: 516.901.1720
>>
>>
>> On 12/18/2012 2:58 PM, Adrian Crum wrote:
>>> "Use File Store? True indicates that the cache can grow beyond the
>>> memory limit and will push additional elements to disk. This has
>>> limited usefulness since the processing required to integrate with the
>>> file system is roughly equivalent to retrieving from a database. We
>>> recommend that this is set to false unless there is specific knowledge
>>> or requirements to use a File Store."
>>>
>>> That statement is overly simplistic. It assumes caches are used solely
>>> for storing database data - when they are also used to store OFBiz
>>> internal structures. Caching to disk can reduce the time required to
>>> retrieve/parse many things that are not stored in the database.
>>>
>>> -Adrian
>>>
>>> On 12/18/2012 3:39 PM, Nick Rosser wrote:
>>>> This is a guide that we put together as we were coming to terms with
>>>> the OFBiz caching and memory scheme. Existing documentation, as far
>>>> as we could tell, was just about non-existent. Since the caching
>>>> scheme is so important from a performance and memory usage point of
>>>> view we decided to document our findings.
>>>>
>>>> For anyone's review -- we'd be interested in any feedback:
>>>>
>>>> http://bigfish.salmonllc.com/help/guideCachingAndMemory.htm
>>>>