entity-one no longer converting types (probably a GenericEntity.setAllFields issue)

[Adrian Crum]

David E Jones wrote:

> On Feb 24, 2010, at 3:01 PM, Adrian Crum wrote:
>
>> David E Jones wrote:
>>> On Feb 24, 2010, at 1:15 PM, Adrian Crum wrote:
>>>> David E Jones wrote:
>>>>> I'm wondering though: how many more places are not handling the time zone correctly? Is there a better way to manage things when the user's time zone setting is different from the server's? Right now it's a mess...
>>>> This will be a worthwhile discussion - because handling time zones correctly hasn't been discussed before. First off: What does handling them correctly mean?
>>>>
>>>> Right now, the process is to try to use the user's time zone if it is available, otherwise use the default. There is no strategy or enforcement of a set of rules - so it isn't surprising that there are inconsistencies.
>>> I'll step back first and describe briefly what the goal of time zone support seems to be: let the user see all times in their own time zone regardless of the server's time zone. This is complicated because when the server converts a Timestamp (or java.util.Date) to a String it does so with either the server's time zone (by default), or by the user's time zone if we do that explicitly. When a Timestamp String comes from a user, whether it be something the user entered by hand or something that is a parameter that is just passed through the browser to another request (in a URL parameter or a hidden form field), the server can interpret that in the user's time zone if there is one, or in the server's time zone by default. In other words, these String representations of Timestamps passed through the browser need to be interpreted consistently because they do not carry time zone information with them (ie they are not like the Timestamp internal representation which is a relat

ive time from a constant point in time, in the GMT time zone so there is no confusion).

>>> So, we have consistency problems on two sides:
>>> 1. producing timestamp Strings for users to, or for the browser to pass back to the system in a URL parameter or hidden field
>>> 2. interpreting timestamp Strings in HTTP requests, from form fields or URL parameters
>>> On both sides we have issues where changing between String value and Timestamp object may be done in the server's time zone or the user's time zone.
>>> It would be nice if all of the code that produced the Strings or consumed them was consistent, but it is spread all over and not consistent.
>> Plus, how do you specify a time zone and where? What if you have multi-tenancy running on a single OFBiz instance and each tenant (or store) wants to appear to be in a different time zone? That's even more complicated than server versus user time zones.
>>
>> The fundamental problem is there is no direct relation of any given Timestamp to any TimeZone.
>
> That's not correct. The Timestamp object has an internal representation that is relative to GMT. Or, more to the point, the internal representation is just a long that is relative to a certain point in time so time zone doesn't matter.
>
> The problem comes into play when you want to convert it to or from a string that consists of things like days and hours.

Exactly. Like I said: "The fundamental problem is there is no direct
relation of any given Timestamp to any TimeZone." You have to have
*both* to convert a Timestamp to a String.

You have a Timestamp instance in your hand and you need to convert it to
a String. What time zone do you use? Who knows? There isn't enough
information in the data type to do the conversion.

>>> So, how to fix it:
>>> 1. more testing where the user's time zone is different from the server's, and fix one problem at a time
>>> 2. try to introduce some sort of time zone value in timestamp String representations so that we know what the time zone is instead of trying to stay consistent with either the user's time zone or the server's
>>> 3. refactor all Timestamp code deal with #1 or #2 above to go through a single spot and do the time zone compensation consistently
>> 4. Use stronger typing. Create a LocalizedDate class and pass that around instead of a plain Date or Timestamp. Then it will be clear what the intention is. A plain Timestamp would imply the server's time zone.
>
> And how do we pass this object in a URL parameter, or in a hidden HTML form field?

The conversion framework will take care of that. It might be better to
think about application scenarios.

Scenario 1: A user types a date/time into a data entry field and clicks
Submit. The application's design specifies the date/time is referenced
to the user's time zone. So, the date/time string is combined with the
user's time zone in a LocalizedDate and that object is passed around the
framework. The framework code doesn't have to guess what time zone to
use for conversions - because the time zone is included in the
LocalizedDate.

Scenario 2: A Timestamp entity field needs to be displayed. The
application's design specifies the Timestamp is referenced to the
server's time zone. The Timestamp field is combined with the server's
time zone in a LocalizedDate and that object is passed around the
framework. The framework code doesn't have to guess what time zone to
use for conversions - because the time zone is included in the
LocalizedDate.

[David E. Jones-2]

We should probably continue discussing this topic, but FYI the particular problem I found was caused be a bug in the PrimaryKeyFinder code in the way it was setting up the context for the conversion routine. The lower level code was assuming it would be in the context, but the context had already been reduced to just the fields needed for the query.

A fix is in rev 916055.

-David


On Feb 24, 2010, at 1:05 PM, David E Jones wrote:

>
> An update on this problem: It looks like it is another time zone issue, or in other words another spot that doesn't manage time zones consistently (I won't say correctly, because I'm not sure if that applies here... partly because I'm not sure if I like the way that OFBiz handles time zones right now).
>
> In any case, for demonstration purposes here is code that works:
>
> <set field="fromDate" from-field="parameters.fromDate" type="Timestamp"/>
> <entity-one entity-name="ProductFeatureAndAppl" value-field="productFeatureAndAppl" auto-field-map="false">
>    <field-map field-name="productId" from-field="parameters.productId"/>
>    <field-map field-name="productFeatureId" from-field="parameters.productFeatureId"/>
>    <field-map field-name="fromDate" from-field="fromDate"/>
> </entity-one>
>
> Here is code that does not work, but should:
>
> <entity-one entity-name="ProductFeatureAndAppl" value-field="productFeatureAndAppl" auto-field-map="false">
>    <field-map field-name="productId" from-field="parameters.productId"/>
>    <field-map field-name="productFeatureId" from-field="parameters.productFeatureId"/>
>    <field-map field-name="fromDate" from-field="parameters.fromDate"/>
> </entity-one>
>
> This also does not work, but should:
>
> <entity-one entity-name="ProductFeatureAndAppl" value-field="productFeatureAndAppl" auto-field-map="true"/>
>
> The problem turns out to not be with the data type, but with the time zone being treated differently in different cases.
>
> I'll work on getting both of these to do the server-side time zone compensation the same as the first one.
>
> I'm wondering though: how many more places are not handling the time zone correctly? Is there a better way to manage things when the user's time zone setting is different from the server's? Right now it's a mess...
>
> -David
>
>
> On Feb 23, 2010, at 10:03 PM, David E Jones wrote:
>
>>
>> What's strange is that this was working fine, even on MySQL, the last time I played with it. Unfortunately that was a bit of time ago so I don't have a good reference to narrow this down.
>>
>> I'll try some things on Derby and see if I can reproduce it, and with more detailed steps to do so.
>>
>> -David
>>
>>
>> On Feb 23, 2010, at 9:49 PM, Adrian Crum wrote:
>>
>>> I was unable to reproduce the problem in the Example component using Derby.
>>>
>>> -Adrian
>>>
>>> --- On Tue, 2/23/10, David E Jones <[hidden email]> wrote:
>>>
>>>> From: David E Jones <[hidden email]>
>>>> Subject: entity-one no longer converting types (probably a GenericEntity.setAllFields issue)
>>>> To: [hidden email]
>>>> Date: Tuesday, February 23, 2010, 8:32 PM
>>>>
>>>> Has anyone made any changes GenericEntity.setAllFields
>>>> method or the code underneath it? It no longer works unless
>>>> the primary key consists only of String type fields.
>>>>
>>>> This is easy to reproduce, even in the Example Application,
>>>> by trying to work with ExampleFeatureAppl records, which
>>>> have a fromDate as part of the primary key.
>>>>
>>>> This warning is showing, and the underlying code does not
>>>> work (on MySQL at least):
>>>>
>>>>
>>>>    [java] 2010-02-23 21:22:42,706
>>>> (http-0.0.0.0-8443-1) [    
>>>> GenericEntity.java:422:WARN ]
>>>>    [java] ---- exception report
>>>> ----------------------------------------------------------
>>>>    [java] =-=-=-=-=-=-=-=-= Database
>>>> type warning GenericEntity.set =-=-=-=-=-=-=-=-= In entity
>>>> field [ProductFeatureAndAppl.fromDate] set the value passed
>>>> in [java.lang.String] is not compatible with the Java type
>>>> of the field [java.sql.Timestamp]
>>>>    [java] Exception:
>>>> java.lang.Exception
>>>>    [java] Message: Location of
>>>> database type warning
>>>>    [java] ---- stack trace
>>>> ---------------------------------------------------------------
>>>>    [java] java.lang.Exception:
>>>> Location of database type warning
>>>>    [java]
>>>> org.ofbiz.entity.GenericEntity.set(GenericEntity.java:422)
>>>>    [java]
>>>> org.ofbiz.entity.GenericEntity.set(GenericEntity.java:368)
>>>>    [java]
>>>> org.ofbiz.entity.GenericEntity.setAllFields(GenericEntity.java:885)
>>>>    [java]
>>>> org.ofbiz.entity.finder.PrimaryKeyFinder.runFind(PrimaryKeyFinder.java:105)
>>>>    [java]
>>>> org.ofbiz.entity.finder.PrimaryKeyFinder.runFind(PrimaryKeyFinder.java:81)
>>>>    [java]
>>>> org.ofbiz.widget.screen.ModelScreenAction$EntityOne.runAction(ModelScreenAction.java:523)
>>>>
>>>> -David
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>
>

[Adam Heath-2]

David E Jones wrote:
> We should probably continue discussing this topic, but FYI the particular problem I found was caused be a bug in the PrimaryKeyFinder code in the way it was setting up the context for the conversion routine. The lower level code was assuming it would be in the context, but the context had already been reduced to just the fields needed for the query.
>
> A fix is in rev 916055.

I see what you did there, and it makes sense.  How did this work
before the conversion system was created, when FSE and FMA were being
used, both of which eventually end up calling
ObjectType.simpleTypeConvert?  Did you investigate that far?  I'm curious.