Update: Unified Expression Language Integration

[Adrian Crum]

Just a quick update on this in case anyone is wondering what happened to
it (or me)...

I've been working full time on improving the UEL integration ever since
the big discussion on the ml the other day. I hope to get this work
committed soon, but I'm taking my time to make sure all of the "gotchas"
have been found and dealt with, and to get some good testing done.

I know some are trying to fix UEL-generated problems by coming up with
unusual syntax - which is fine for fixing something broken. But I hope
we can hold off on using unusual syntax as a general rule until I get
this work committed - which will eliminate the need for most of it.

Some of the things I've been working on:

1. Getting the JUEL library to support the OFBiz List syntax -
someList[+0], someList[], etc. I have that working now. Internally, the
syntax is converted: someList[+0] becomes someList['insert@0'] and
someList[] becomes someList['add'].

2. Getting the JUEL library to support the on-the-fly object creation
used in OFBiz syntax. In OFBiz, setting parameters.someMap.someElement
to some value will create the "parameters" Map and "someMap" Map if they
don't exist. According to the UEL spec, values can only be set on
existing variables. The OFBiz syntax was a challenge to implement, but I
have it working now.

3. Getting the FlexibleMapAccessor and FlexibleStringExpander to handle
expressions the same. For those who don't know, FlexibleMapAccessor is
used for l-values, and FlexibleStringExpander is used for r-values. Each
class has its own home-grown parser and as a result, they support
different syntaxes. I'd like to get both classes to use the JUEL library
so that they both handle UEL the same - maybe even combine them into a
single class. I'm working on this step now.

-Adrian

[Adam Heath-2]

Adrian Crum wrote:

> 2. Getting the JUEL library to support the on-the-fly object creation
> used in OFBiz syntax. In OFBiz, setting parameters.someMap.someElement
> to some value will create the "parameters" Map and "someMap" Map if they
> don't exist. According to the UEL spec, values can only be set on
> existing variables. The OFBiz syntax was a challenge to implement, but I
> have it working now.

This is called auto-vivify.

[Adrian Crum]

Cool. I have a new buzzword.

Adam Heath wrote:

> Adrian Crum wrote:
>
>> 2. Getting the JUEL library to support the on-the-fly object creation
>> used in OFBiz syntax. In OFBiz, setting parameters.someMap.someElement
>> to some value will create the "parameters" Map and "someMap" Map if they
>> don't exist. According to the UEL spec, values can only be set on
>> existing variables. The OFBiz syntax was a challenge to implement, but I
>> have it working now.
>
> This is called auto-vivify.
>
>

[Adam Heath-2]

Adrian Crum wrote:
> Cool. I have a new buzzword.

Actually, remove the -, it's autovivify, and it's a perl thing.

[David E Jones-3]

Thanks for the update Adrian. As I mentioned before, this is a really  
nice thing to get into the project.

BTW, the FlexibleMapAccessor and FlexibleStringExpander should not  
have independent parsers, unless someone has changed that since I  
wrote them. The FlexibleStringExpander should use the  
FlexibleMapAccessor whenever accessing a variable, but has additional  
syntax to support like the ${} for actual string expansion. If that  
isn't the case anymore... that's bad.

Either way, doing things with UEL may be a fair bit different and as  
long as we have different ways of using JUEL for different intended  
expressions (like identifying variables as a target or l-value, versus  
expansion to an object to do something else with or r-value) then  
we're fine.

-David


On Dec 18, 2008, at 3:03 PM, Adrian Crum wrote:

> Just a quick update on this in case anyone is wondering what  
> happened to it (or me)...
>
> I've been working full time on improving the UEL integration ever  
> since the big discussion on the ml the other day. I hope to get this  
> work committed soon, but I'm taking my time to make sure all of the  
> "gotchas" have been found and dealt with, and to get some good  
> testing done.
>
> I know some are trying to fix UEL-generated problems by coming up  
> with unusual syntax - which is fine for fixing something broken. But  
> I hope we can hold off on using unusual syntax as a general rule  
> until I get this work committed - which will eliminate the need for  
> most of it.
>
> Some of the things I've been working on:
>
> 1. Getting the JUEL library to support the OFBiz List syntax -  
> someList[+0], someList[], etc. I have that working now. Internally,  
> the syntax is converted: someList[+0] becomes someList['insert@0']  
> and someList[] becomes someList['add'].
>
> 2. Getting the JUEL library to support the on-the-fly object  
> creation used in OFBiz syntax. In OFBiz, setting  
> parameters.someMap.someElement to some value will create the  
> "parameters" Map and "someMap" Map if they don't exist. According to  
> the UEL spec, values can only be set on existing variables. The  
> OFBiz syntax was a challenge to implement, but I have it working now.
>
> 3. Getting the FlexibleMapAccessor and FlexibleStringExpander to  
> handle expressions the same. For those who don't know,  
> FlexibleMapAccessor is used for l-values, and FlexibleStringExpander  
> is used for r-values. Each class has its own home-grown parser and  
> as a result, they support different syntaxes. I'd like to get both  
> classes to use the JUEL library so that they both handle UEL the  
> same - maybe even combine them into a single class. I'm working on  
> this step now.
>
> -Adrian

[Adrian Crum]

You are correct, FlexibleStringExpander called FlexibleMapAccessor to
expand variables. The UEL commit changed that to have JUEL expand
FlexibleStringExpander variables.

FlexibleMapAccessor doesn't parse ${} expressions - so there are places
where classes have instances of both FlexibleMapAccessor and
FlexibleStringExpander. That's what I meant about getting the two to
parse in the same way - one class should (in my opinion) handle all
expressions.

JUEL will throw an exception if you try to assign a value to an r-value
- so we're covered there.

-Adrian


David E Jones wrote:

>
> Thanks for the update Adrian. As I mentioned before, this is a really
> nice thing to get into the project.
>
> BTW, the FlexibleMapAccessor and FlexibleStringExpander should not have
> independent parsers, unless someone has changed that since I wrote them.
> The FlexibleStringExpander should use the FlexibleMapAccessor whenever
> accessing a variable, but has additional syntax to support like the ${}
> for actual string expansion. If that isn't the case anymore... that's bad.
>
> Either way, doing things with UEL may be a fair bit different and as
> long as we have different ways of using JUEL for different intended
> expressions (like identifying variables as a target or l-value, versus
> expansion to an object to do something else with or r-value) then we're
> fine.
>
> -David
>
>
> On Dec 18, 2008, at 3:03 PM, Adrian Crum wrote:
>
>> Just a quick update on this in case anyone is wondering what happened
>> to it (or me)...
>>
>> I've been working full time on improving the UEL integration ever
>> since the big discussion on the ml the other day. I hope to get this
>> work committed soon, but I'm taking my time to make sure all of the
>> "gotchas" have been found and dealt with, and to get some good testing
>> done.
>>
>> I know some are trying to fix UEL-generated problems by coming up with
>> unusual syntax - which is fine for fixing something broken. But I hope
>> we can hold off on using unusual syntax as a general rule until I get
>> this work committed - which will eliminate the need for most of it.
>>
>> Some of the things I've been working on:
>>
>> 1. Getting the JUEL library to support the OFBiz List syntax -
>> someList[+0], someList[], etc. I have that working now. Internally,
>> the syntax is converted: someList[+0] becomes someList['insert@0'] and
>> someList[] becomes someList['add'].
>>
>> 2. Getting the JUEL library to support the on-the-fly object creation
>> used in OFBiz syntax. In OFBiz, setting parameters.someMap.someElement
>> to some value will create the "parameters" Map and "someMap" Map if
>> they don't exist. According to the UEL spec, values can only be set on
>> existing variables. The OFBiz syntax was a challenge to implement, but
>> I have it working now.
>>
>> 3. Getting the FlexibleMapAccessor and FlexibleStringExpander to
>> handle expressions the same. For those who don't know,
>> FlexibleMapAccessor is used for l-values, and FlexibleStringExpander
>> is used for r-values. Each class has its own home-grown parser and as
>> a result, they support different syntaxes. I'd like to get both
>> classes to use the JUEL library so that they both handle UEL the same
>> - maybe even combine them into a single class. I'm working on this
>> step now.
>>
>> -Adrian
>
>

[Adrian Crum]

The work continues...

Good news! The JUEL library author has been working with me to make
changes to JUEL so that the auto-vivify extension is easier to
implement. He committed the changes Sunday and I tested them today.

JUEL is throwing exceptions when we try to assign values to null. OFBiz
syntax allows a value to be called "null" - but that causes problems
with third party libraries that expect it to be a literal value (or
r-value). Another example is FreeMarker - see the comment on line 189 of
FreeMarkerWorker.java. We should start using a different identifier
(like "null-value") that will cause fewer conflicts. I'm working on
treating this like I did the OFBiz List syntax - convert it internally
to something else.

-Adrian

Adrian Crum wrote:

> Just a quick update on this in case anyone is wondering what happened to
> it (or me)...
>
> I've been working full time on improving the UEL integration ever since
> the big discussion on the ml the other day. I hope to get this work
> committed soon, but I'm taking my time to make sure all of the "gotchas"
> have been found and dealt with, and to get some good testing done.
>
> I know some are trying to fix UEL-generated problems by coming up with
> unusual syntax - which is fine for fixing something broken. But I hope
> we can hold off on using unusual syntax as a general rule until I get
> this work committed - which will eliminate the need for most of it.
>
> Some of the things I've been working on:
>
> 1. Getting the JUEL library to support the OFBiz List syntax -
> someList[+0], someList[], etc. I have that working now. Internally, the
> syntax is converted: someList[+0] becomes someList['insert@0'] and
> someList[] becomes someList['add'].
>
> 2. Getting the JUEL library to support the on-the-fly object creation
> used in OFBiz syntax. In OFBiz, setting parameters.someMap.someElement
> to some value will create the "parameters" Map and "someMap" Map if they
> don't exist. According to the UEL spec, values can only be set on
> existing variables. The OFBiz syntax was a challenge to implement, but I
> have it working now.
>
> 3. Getting the FlexibleMapAccessor and FlexibleStringExpander to handle
> expressions the same. For those who don't know, FlexibleMapAccessor is
> used for l-values, and FlexibleStringExpander is used for r-values. Each
> class has its own home-grown parser and as a result, they support
> different syntaxes. I'd like to get both classes to use the JUEL library
> so that they both handle UEL the same - maybe even combine them into a
> single class. I'm working on this step now.
>
> -Adrian
>

[David E Jones-3]

On Dec 22, 2008, at 1:11 PM, Adrian Crum wrote:

> JUEL is throwing exceptions when we try to assign values to null.  
> OFBiz syntax allows a value to be called "null" - but that causes  
> problems with third party libraries that expect it to be a literal  
> value (or r-value). Another example is FreeMarker - see the comment  
> on line 189 of FreeMarkerWorker.java. We should start using a  
> different identifier (like "null-value") that will cause fewer  
> conflicts. I'm working on treating this like I did the OFBiz List  
> syntax - convert it internally to something else.

We should definitely get rid of the concept of an object called  
"null". Now that we have a real expression evaluator in place there is  
no need for it. We do still have objects like "nullField" floating  
around, but I don't know that we really need other ones.

Of course, you're the one digging into the details on this... if you  
see a need for a more generic null object let's chat about it. We  
probably wouldn't want to call it "null-value" as that wold introduce  
another subtraction operator issue... ;)

-David

[Adrian Crum]

Thanks for the input! I ended up returning the null accessor instance
for expression == "null" and it seems to work okay.

I just need to work out a small bug with the LocalizedMap handling and
then I should be ready to commit this.

-Adrian

David E Jones wrote:

>
> On Dec 22, 2008, at 1:11 PM, Adrian Crum wrote:
>
>> JUEL is throwing exceptions when we try to assign values to null.
>> OFBiz syntax allows a value to be called "null" - but that causes
>> problems with third party libraries that expect it to be a literal
>> value (or r-value). Another example is FreeMarker - see the comment on
>> line 189 of FreeMarkerWorker.java. We should start using a different
>> identifier (like "null-value") that will cause fewer conflicts. I'm
>> working on treating this like I did the OFBiz List syntax - convert it
>> internally to something else.
>
> We should definitely get rid of the concept of an object called "null".
> Now that we have a real expression evaluator in place there is no need
> for it. We do still have objects like "nullField" floating around, but I
> don't know that we really need other ones.
>
> Of course, you're the one digging into the details on this... if you see
> a need for a more generic null object let's chat about it. We probably
> wouldn't want to call it "null-value" as that wold introduce another
> subtraction operator issue... ;)
>
> -David
>
>

[Adrian Crum-2]

An improved UEL implementation was committed - rev 729583. Here is a summary of what is implemented (some of this has been mentioned before):

1. OFBiz scripting now fully supports the Unified Expression Language.

2. The UEL has been extended to support OFBiz syntax:
  a. [+integer] to insert List items at a specified index
  b. [] to add items to the end of a List
  c. Auto-vivify (automatic variable creation)

3. The implementation converts the [+integer] and [] syntax internally to ['insert@integer'] and ['add']. The internal syntax could be used in the scripts as well - I'll leave it up to the community to decide if we should do that.

The implementation is still asymmetric - l-values cannot contain nested expressions, but r-values can. I'm working on changing that.

-Adrian

--- On Mon, 12/22/08, Adrian Crum <[hidden email]> wrote:

> From: Adrian Crum <[hidden email]>
> Subject: Re: Update: Unified Expression Language Integration
> To: [hidden email]
> Date: Monday, December 22, 2008, 12:11 PM
> The work continues...
>
> Good news! The JUEL library author has been working with me
> to make changes to JUEL so that the auto-vivify extension is
> easier to implement. He committed the changes Sunday and I
> tested them today.
>
> JUEL is throwing exceptions when we try to assign values to
> null. OFBiz syntax allows a value to be called
> "null" - but that causes problems with third party
> libraries that expect it to be a literal value (or r-value).
> Another example is FreeMarker - see the comment on line 189
> of FreeMarkerWorker.java. We should start using a different
> identifier (like "null-value") that will cause
> fewer conflicts. I'm working on treating this like I did
> the OFBiz List syntax - convert it internally to something
> else.
>
> -Adrian
>
> Adrian Crum wrote:
> > Just a quick update on this in case anyone is
> wondering what happened to it (or me)...
> >
> > I've been working full time on improving the UEL
> integration ever since the big discussion on the ml the
> other day. I hope to get this work committed soon, but
> I'm taking my time to make sure all of the
> "gotchas" have been found and dealt with, and to
> get some good testing done.
> >
> > I know some are trying to fix UEL-generated problems
> by coming up with unusual syntax - which is fine for fixing
> something broken. But I hope we can hold off on using
> unusual syntax as a general rule until I get this work
> committed - which will eliminate the need for most of it.
> >
> > Some of the things I've been working on:
> >
> > 1. Getting the JUEL library to support the OFBiz List
> syntax - someList[+0], someList[], etc. I have that working
> now. Internally, the syntax is converted: someList[+0]
> becomes someList['insert@0'] and someList[] becomes
> someList['add'].
> >
> > 2. Getting the JUEL library to support the on-the-fly
> object creation used in OFBiz syntax. In OFBiz, setting
> parameters.someMap.someElement to some value will create the
> "parameters" Map and "someMap" Map if
> they don't exist. According to the UEL spec, values can
> only be set on existing variables. The OFBiz syntax was a
> challenge to implement, but I have it working now.
> >
> > 3. Getting the FlexibleMapAccessor and
> FlexibleStringExpander to handle expressions the same. For
> those who don't know, FlexibleMapAccessor is used for
> l-values, and FlexibleStringExpander is used for r-values.
> Each class has its own home-grown parser and as a result,
> they support different syntaxes. I'd like to get both
> classes to use the JUEL library so that they both handle UEL
> the same - maybe even combine them into a single class.
> I'm working on this step now.
> >
> > -Adrian
> >