Supporting "null" in data load update

[Bob Morley]

We ran into a situation where we had a previously created seeded entity who had a field value that we wanted to change to null.  The current data load process would take values from the data files and if the field value was set to either empty string OR was not present, would ignore this field attribute.  The net result, was that there was no way to get "null" populated on the entity via the standard data load update mechansim.

As a result, I created this ticket OFBIZ-3421 with a small fix that would treat something like :

    <ProductSubscriptionResource
        productId="DNSZONE" subscriptionResourceId="DNSZONE" fromDate="2009-01-01 00:00:00.0"
    useTime="1" useTimeUomId="TF_mon"
        useCountLimit="null"
    canclAutmExtTime="1" canclAutmExtTimeUomId="TF_day"
    useRoleTypeId="CUSTOMER"
    automaticExtend="Y"/>

As updating the useCountLimit to null on update.  Upon further discussion, the thought was that perhaps we should change how Ofbiz treats elements that are explicitly set to "" so that they are treated as null in general (rather than being ignored like they are not specified at all).

What I am saying here is the current data load would treat useCountLimit="" exactly the same as if useCountLimit was not specified at all.  And we are suggesting that useCountLimit="" should really be treated as updating to null so that a previous value would be reset to null.

That would prevent the "hack-like" feeling I am getting from treating "null" as some sort of keyword which is what I had done in this resolution.

The net result of this, is that we would likely have to blow through about 1400 instances of *Data.xml files that are explicitly setting fields to ="" since it is likely that they really want them ignored.

[Scott Gray-2]

Hi Bob,

I was under the impression that XML readers tend to treat a missing attribute the same as an empty attribute i.e. there is no way to tell the difference between them.  Is that not the case?

Regards
Scott

HotWax Media
http://www.hotwaxmedia.com

On 22/01/2010, at 9:30 AM, Bob Morley wrote:

>
> We ran into a situation where we had a previously created seeded entity who
> had a field value that we wanted to change to null.  The current data load
> process would take values from the data files and if the field value was set
> to either empty string OR was not present, would ignore this field
> attribute.  The net result, was that there was no way to get "null"
> populated on the entity via the standard data load update mechansim.
>
> As a result, I created this ticket OFBIZ-3421 with a small fix that would
> treat something like :
>
>    <ProductSubscriptionResource
>        productId="DNSZONE" subscriptionResourceId="DNSZONE"
> fromDate="2009-01-01 00:00:00.0"
>     useTime="1" useTimeUomId="TF_mon"
>        useCountLimit="null"
>     canclAutmExtTime="1" canclAutmExtTimeUomId="TF_day"
>     useRoleTypeId="CUSTOMER"
>     automaticExtend="Y"/>
>
> As updating the useCountLimit to null on update.  Upon further discussion,
> the thought was that perhaps we should change how Ofbiz treats elements that
> are explicitly set to "" so that they are treated as null in general (rather
> than being ignored like they are not specified at all).
>
> What I am saying here is the current data load would treat useCountLimit=""
> exactly the same as if useCountLimit was not specified at all.  And we are
> suggesting that useCountLimit="" should really be treated as updating to
> null so that a previous value would be reset to null.
>
> That would prevent the "hack-like" feeling I am getting from treating "null"
> as some sort of keyword which is what I had done in this resolution.
>
> The net result of this, is that we would likely have to blow through about
> 1400 instances of *Data.xml files that are explicitly setting fields to =""
> since it is likely that they really want them ignored.
> --
> View this message in context: http://n4.nabble.com/Supporting-null-in-data-load-update-tp1100195p1100195.html
> Sent from the OFBiz - Dev mailing list archive at Nabble.com.

[Adrian Crum]

Plus, I like being specific - using fieldName="null" makes it clear what
you intend to do.

-Adrian

Scott Gray wrote:

> Hi Bob,
>
> I was under the impression that XML readers tend to treat a missing attribute the same as an empty attribute i.e. there is no way to tell the difference between them.  Is that not the case?
>
> Regards
> Scott
>
> HotWax Media
> http://www.hotwaxmedia.com
>
> On 22/01/2010, at 9:30 AM, Bob Morley wrote:
>
>> We ran into a situation where we had a previously created seeded entity who
>> had a field value that we wanted to change to null.  The current data load
>> process would take values from the data files and if the field value was set
>> to either empty string OR was not present, would ignore this field
>> attribute.  The net result, was that there was no way to get "null"
>> populated on the entity via the standard data load update mechansim.
>>
>> As a result, I created this ticket OFBIZ-3421 with a small fix that would
>> treat something like :
>>
>>    <ProductSubscriptionResource
>>        productId="DNSZONE" subscriptionResourceId="DNSZONE"
>> fromDate="2009-01-01 00:00:00.0"
>>     useTime="1" useTimeUomId="TF_mon"
>>        useCountLimit="null"
>>     canclAutmExtTime="1" canclAutmExtTimeUomId="TF_day"
>>     useRoleTypeId="CUSTOMER"
>>     automaticExtend="Y"/>
>>
>> As updating the useCountLimit to null on update.  Upon further discussion,
>> the thought was that perhaps we should change how Ofbiz treats elements that
>> are explicitly set to "" so that they are treated as null in general (rather
>> than being ignored like they are not specified at all).
>>
>> What I am saying here is the current data load would treat useCountLimit=""
>> exactly the same as if useCountLimit was not specified at all.  And we are
>> suggesting that useCountLimit="" should really be treated as updating to
>> null so that a previous value would be reset to null.
>>
>> That would prevent the "hack-like" feeling I am getting from treating "null"
>> as some sort of keyword which is what I had done in this resolution.
>>
>> The net result of this, is that we would likely have to blow through about
>> 1400 instances of *Data.xml files that are explicitly setting fields to =""
>> since it is likely that they really want them ignored.
>> --
>> View this message in context: http://n4.nabble.com/Supporting-null-in-data-load-update-tp1100195p1100195.html
>> Sent from the OFBiz - Dev mailing list archive at Nabble.com.
>

[Adam Heath-2]

Adrian Crum wrote:
> Plus, I like being specific - using fieldName="null" makes it clear what
> you intend to do.

="null" is a string, who's contents are null.  There can be no other
way to interpet that.

Better would be to do null_$field="", or some such.  You can
definately tell when an attribute exists, vs. just being an empty
value.  You have to used the NamedNodeMap interface, looping over the
attributes.

[Bob Morley]

Actually I just did a quick test and the SAX parser returns all attributes that are defined in the document (even if they have empty string as their value).  The code as originally written ...

                        // treat empty strings as nulls
                        if (value != null && value.length() > 0) {

Exclude if the value is null (not sure how that would happen) but also if it is empty string.  So it would be technically feasible to treat explicit empty string values as "null" for the database and having non-specified field values ignored (as they are now).

Having said that, if we feel it is more reasonable to be explicit here (fieldname="null" or null_fieldname="") then I am all for it.

It seems to me personally, that it is more intuitive to have fieldname="" imply an explicit set to null in the database which would be consist with what you get on the initial create and would be a force back to "blank" on an update.  Not having the field clearly implies ignore (which is what it does now), etc etc etc.

Anyone buying that?  :)

[Adam Heath-2]

Bob Morley wrote:

> Actually I just did a quick test and the SAX parser returns all attributes
> that are defined in the document (even if they have empty string as their
> value).  The code as originally written ...
>
>                         // treat empty strings as nulls
>                         if (value != null && value.length() > 0) {
>
> Exclude if the value is null (not sure how that would happen) but also if it
> is empty string.  So it would be technically feasible to treat explicit
> empty string values as "null" for the database and having non-specified
> field values ignored (as they are now).

Again, no.  There is a very big difference between an actual empty
string, and a null value.  Do not overload the meaning.

> Having said that, if we feel it is more reasonable to be explicit here
> (fieldname="null" or null_fieldname="") then I am all for it.
>
> It seems to me personally, that it is more intuitive to have fieldname=""
> imply an explicit set to null in the database which would be consist with
> what you get on the initial create and would be a force back to "blank" on
> an update.  Not having the field clearly implies ignore (which is what it
> does now), etc etc etc.
>
> Anyone buying that?  :)

Not me.

[Jacques Le Roux]

Just a detail:  this is certainly not in trunk since around r883364 I have replaced all these kinds of idioms by some kinds of
UtilValidate.is*Empty(), here for instance: UtilValidate.isNotEmpty()

Jacques

From: "Adam Heath" <[hidden email]>

> Bob Morley wrote:
>> Actually I just did a quick test and the SAX parser returns all attributes
>> that are defined in the document (even if they have empty string as their
>> value).  The code as originally written ...
>>
>>                         // treat empty strings as nulls
>>                         if (value != null && value.length() > 0) {
>>
>> Exclude if the value is null (not sure how that would happen) but also if it
>> is empty string.  So it would be technically feasible to treat explicit
>> empty string values as "null" for the database and having non-specified
>> field values ignored (as they are now).
>
> Again, no.  There is a very big difference between an actual empty
> string, and a null value.  Do not overload the meaning.
>
>> Having said that, if we feel it is more reasonable to be explicit here
>> (fieldname="null" or null_fieldname="") then I am all for it.
>>
>> It seems to me personally, that it is more intuitive to have fieldname=""
>> imply an explicit set to null in the database which would be consist with
>> what you get on the initial create and would be a force back to "blank" on
>> an update.  Not having the field clearly implies ignore (which is what it
>> does now), etc etc etc.
>>
>> Anyone buying that?  :)
>
> Not me.
>

[Jacques Le Roux]

Ha, I see it's in https://issues.apache.org/jira/browse/OFBIZ-3421

Jacques

From: "Jacques Le Roux" <[hidden email]>

> Just a detail:  this is certainly not in trunk since around r883364 I have replaced all these kinds of idioms by some kinds of
> UtilValidate.is*Empty(), here for instance: UtilValidate.isNotEmpty()
>
> Jacques
>
> From: "Adam Heath" <[hidden email]>
>> Bob Morley wrote:
>>> Actually I just did a quick test and the SAX parser returns all attributes
>>> that are defined in the document (even if they have empty string as their
>>> value).  The code as originally written ...
>>>
>>>                         // treat empty strings as nulls
>>>                         if (value != null && value.length() > 0) {
>>>
>>> Exclude if the value is null (not sure how that would happen) but also if it
>>> is empty string.  So it would be technically feasible to treat explicit
>>> empty string values as "null" for the database and having non-specified
>>> field values ignored (as they are now).
>>
>> Again, no.  There is a very big difference between an actual empty
>> string, and a null value.  Do not overload the meaning.
>>
>>> Having said that, if we feel it is more reasonable to be explicit here
>>> (fieldname="null" or null_fieldname="") then I am all for it.
>>>
>>> It seems to me personally, that it is more intuitive to have fieldname=""
>>> imply an explicit set to null in the database which would be consist with
>>> what you get on the initial create and would be a force back to "blank" on
>>> an update.  Not having the field clearly implies ignore (which is what it
>>> does now), etc etc etc.
>>>
>>> Anyone buying that?  :)
>>
>> Not me.
>>
>
>