Discussion: Expression Cache

[Adrian Crum]

Right now the expression caches have no maximum size or expire time. The
FlexibleStringExpander cache shouldn't be a problem with those settings
because the expressions are like constants - they don't change during
program execution. The maximum size on the demo server runs around
3000-5000 entries, so I think we're safe there.

I'm concerned about the FlexibleMapAccessor cache though. Since IDs are
used to create expressions, there is a potential for that cache to grow
to tens of thousands. I think some kind of limit should be put on that
cache, but I'm not sure what they should be.

Any advice would be appreciated.

-Adrian

[David E Jones-3]

Chances are the size of each cache entry will be pretty small, so even  
with lots of entries the amount of memory shouldn't be too large. In  
general I don't like the idea of a size limited cache that uses and  
LRU algorithm because there is a fair amount of overhead to maintain  
the LRU list every time the cache entry is accessed (in both memory  
and processing cycles).

Either way, it's easy to tune cache in production and hard to guess  
about the impact of caches during development, so I'd say unless we  
had some hard data that shows a memory problem and minimum performance  
and memory impact from the LRU list I'd say we leave the default  
settings as a wide open cache with no size limit and that doesn't have  
a time limit either.

If you're really worried about it you could do something to load all  
existing simple-method, screen, etc files and see what the actual size  
of the cache is. This is really easy to do since if you run the  
Artifact Info stuff (just go to the main page) it will load everything.

-David


On Dec 29, 2008, at 11:25 AM, Adrian Crum wrote:

> Right now the expression caches have no maximum size or expire time.  
> The FlexibleStringExpander cache shouldn't be a problem with those  
> settings because the expressions are like constants - they don't  
> change during program execution. The maximum size on the demo server  
> runs around 3000-5000 entries, so I think we're safe there.
>
> I'm concerned about the FlexibleMapAccessor cache though. Since IDs  
> are used to create expressions, there is a potential for that cache  
> to grow to tens of thousands. I think some kind of limit should be  
> put on that cache, but I'm not sure what they should be.
>
> Any advice would be appreciated.
>
> -Adrian

[Adrian Crum-2]

--- On Tue, 12/30/08, David E Jones <[hidden email]> wrote:
> If you're really worried about it you could do
> something to load all existing simple-method, screen, etc
> files and see what the actual size of the cache is. This is
> really easy to do since if you run the Artifact Info stuff
> (just go to the main page) it will load everything.

That's a great idea! I'll give it a try.

My concern is with simple methods using IDs as identifiers. Let's say an accounting method creates a Map of invoices that need some kind of reconciling done on them. Each invoice ID is a Map key. A high volume user might have 10,000 invoices, so 10,000 expressions are generated and kept in the cache. Then in another simple method, orders are analyzed for backordered items and the same approach is used. There are 10,000 orders so another 10,000 expressions are created and cached. There is no limit to how many places and how many times this happens. I foresee a serious problem.

-Adrian



     

[David E Jones-3]

On Dec 31, 2008, at 8:31 AM, Adrian Crum wrote:

> --- On Tue, 12/30/08, David E Jones <[hidden email]>  
> wrote:
>> If you're really worried about it you could do
>> something to load all existing simple-method, screen, etc
>> files and see what the actual size of the cache is. This is
>> really easy to do since if you run the Artifact Info stuff
>> (just go to the main page) it will load everything.
>
> That's a great idea! I'll give it a try.
>
> My concern is with simple methods using IDs as identifiers. Let's  
> say an accounting method creates a Map of invoices that need some  
> kind of reconciling done on them. Each invoice ID is a Map key. A  
> high volume user might have 10,000 invoices, so 10,000 expressions  
> are generated and kept in the cache. Then in another simple method,  
> orders are analyzed for backordered items and the same approach is  
> used. There are 10,000 orders so another 10,000 expressions are  
> created and cached. There is no limit to how many places and how  
> many times this happens. I foresee a serious problem.

Well, that would be a problem. Previously the expression evaluation  
cached the interpreted code based only on the straight text from the  
file, so no IDs unless they were hard-coded.

Are you saying that in the UEL implementation it does one (inner)  
expansion and then parses the result, and caches that? In other words,  
there is no support for nested expressions?

Ouch. I don't think we can get around that as a limitation... either  
we'll have to somehow get nested expressions working, or just stop  
using nested expressions now that we have alternatives in UEL and we  
really don't need them any more. My vote would be for the second  
option, BTW... much cleaner and may actually be easier.

-David

[Adrian Crum-2]

--- On Wed, 12/31/08, David E Jones <[hidden email]> wrote:

> > --- On Tue, 12/30/08, David E Jones
> <[hidden email]> wrote:
> >> If you're really worried about it you could do
> >> something to load all existing simple-method,
> screen, etc
> >> files and see what the actual size of the cache
> is. This is
> >> really easy to do since if you run the Artifact
> Info stuff
> >> (just go to the main page) it will load
> everything.
> >
> > That's a great idea! I'll give it a try.
> >
> > My concern is with simple methods using IDs as
> identifiers. Let's say an accounting method creates a
> Map of invoices that need some kind of reconciling done on
> them. Each invoice ID is a Map key. A high volume user might
> have 10,000 invoices, so 10,000 expressions are generated
> and kept in the cache. Then in another simple method, orders
> are analyzed for backordered items and the same approach is
> used. There are 10,000 orders so another 10,000 expressions
> are created and cached. There is no limit to how many places
> and how many times this happens. I foresee a serious
> problem.
>
> Well, that would be a problem. Previously the expression
> evaluation cached the interpreted code based only on the
> straight text from the file, so no IDs unless they were
> hard-coded.
>
> Are you saying that in the UEL implementation it does one
> (inner) expansion and then parses the result, and caches
> that? In other words, there is no support for nested
> expressions?
>
> Ouch. I don't think we can get around that as a
> limitation... either we'll have to somehow get nested
> expressions working, or just stop using nested expressions
> now that we have alternatives in UEL and we really don't
> need them any more. My vote would be for the second option,
> BTW... much cleaner and may actually be easier.

I just ran some tests and discovered that my concerns are unfounded. Sorry for the false alarm.

You are correct that we don't need nested expressions anymore. One thing to note is that nested expressions incur a serious performance penalty because the expression is being evaluated more than once. I agree that we should stop using them.

-Adrian