Re: Tasks for a release

[David E Jones-2]

Si Chen wrote:

> Hi everybody -
>
> I'm really glad we will be doing a release for OFBIZ again.  Here are a
> things I would really like to see -
>
> 1.  A release which is on a separate SVN branch with a stable set of
> features, so the incremental releases will be bug fixes that produce
> more stable versions, which would imply:
>
> 2.  That we can coordinate between all the major developers, including
> the committers and any other code contributors, of a feature freeze date
> for doing a release.

With #1 we don't need #2. There is no reason to do a feature freeze, and I don't think the organization of OFBiz is such that trying to do so will be reasonable.

Once a branch is created the procedure from there on out is to merge over important bug fixes from the trunk, and fix bugs that aren't in the trunk right in the branch. Those should be the ONLY two changes going into the branch.

Doing it that way once the branch is created it is automatically a "feature freeze" and so that is not needed in the trunk.

The only time where there might be issues is the period between a release candidate and the final release, but we can address that by starting the branch for the RC? releases and each RC? will correspond to a revision in the branch. The final release will also correspond to a revision in the branch, as will post-final releases based on bugs fixed over time.

So, all we really need is volunteers to fix bugs in the release branches. Doing this job is a LOT of work for a project with the size and regular changes of OFBiz. This is why we do not follow the release early and often approach. If we have too many branches for releases they simply will not be maintained and so that release might as well be an SVN rev number on a sticky note as none of the benefits of a release will apply.

BTW, I'm moving this discussion (starting with this reply) to the ofbiz-dev list. Please reply there for this and other messages in this thread (ie let's move the thread over as this should be a public discussion).

-David

[David E Jones-2]

Si Chen wrote:
> Hi everybody -
>
> I'm really glad we will be doing a release for OFBIZ again.  Here are a
> things I would really like to see -

[snip]

> 3.  Separately, there are some production-readiness issues which we
> should address prior to a release.  For example, there was a typecasting
> issue for the minilang field-to-field and a PostgreSQL bytea/oid issue
> that conflicted with Derby which were never satisfactorily resolved.  I
> think if we're going to put out a release and tell our users "this is
> ready for production use", we should address those.
>
> 4.  Also, has the new Geronimo transaction manager been fully tested?  
> Are the reports of transaction time out issues real?

With the branch process I mentioned in the other email these issues don't matter so much.

Yes, we want to have something as close to production-ready as possible, but OFBiz is a big system that supports a really wide variety of lower lever software to interact with, so there is no way we can guarantee (unless someone has a bunch of money socked away that they want to dedicate to a testing lab...) that OFBiz will work OOTB with all combinations of infrastructure software without subtle configuration and/or code changes specific to a circumstance.

So, while these two issues are worth keeping an eye on and fixing when we can, I don't think either is a show-stopper for a release, even a "production stable" release.

For both of these it may be that those who are able to fix such things don't have the resources to do so now, but any reasonable size deployment team usually has people around who can. Also, such things are often easier to fix for a specific circumstance than something that is meant to work in a wide variety of cases, which is the case with the Postgres issue. With a little creativity and experience it is possible to fix in a very widely applicable way any such issue and not have any problems with Derby or Postgres, but with no resources to do so... the best we can do is get a release out and attract more resources to the community...

On the Geronimo issue: I still haven't seen reports of anything that seems to be a core issue with Geronimo. People report errors all the time that are caused by things like timeouts and such or a lower level error that causes rollback-only to be set and these manifest higher up as transaction errors, but the details are in the logs. So far I haven't seen any clear definition of a problem with Geronimo. That doesn't mean there isn't one, it just means that from what I've seen on the lists it hasn't been isolated yet (ie seen by someone who can recognize it for what it is and distinguish between a real Geronimo problem and a simple timeout or other error that caused the transaction error).

-David

[Yoav Shapira-2]

Hi,

> > You have to mention every externally-developed component and its
> > license.  If an external component has its own license file (I expect
> > most would), you have to include that license file as well, in the
> > same directory as the component itself (e.g. if it's a jar called
> > some-component-1.0.jar, put a some-component-1.0-license.txt in that
> > directory).
>
> Is this necessary for jar files that have a license inside them?

No, JARs that have the license inside are fine by themselves.  So your
past policy is also OK.  Thanks for clarifying,

Yoav

[Si Chen-2]

David,

Without coordinating a feature freeze between the major contributors,  
it would be very very difficult, especially for less experienced  
"volunteers", to maintain the release branches and fix the bugs.  
 From my personal experience trying to create the opentaps releases,  
a good release can only be created if the original version is  
reasonably stable and if the core developers significantly support  
the effort by helping to push the bug fixes from trunk to the release  
branch.

On the issue of stability:

1.  I propose that we put this http://jira.undersunconsulting.com/ 
browse/OFBIZ-500 back into the main code base.  It addressed a  
typecast issue with field-to-field.

2.  I think we should take a vote: how many people would like to keep  
current code "as is", so the "OID" data type (used for storing images  
and content) works with Derby and not PostgreSQL, versus making a  
change which would make it work with PostgreSQL and not Derby?

3.  I'll just keep my fingers crossed about the Geronimo transactions  
manager then.

Si


On Jul 24, 2006, at 10:13 AM, David E Jones wrote:

>

[cjhowe]

Are there any open source projects that are not driven
by a single company that successfully implement
feature freeze releases AND would have the complexity
of feature advancements that OFBiz does (this would
exclude most, if not all, Apache projects as most of
them are one trick ponies, so to speak)?  If there
are, maybe we should see how they best accopmlish
this.

--- Si Chen <[hidden email]> wrote:

> David,
>
> Without coordinating a feature freeze between the
> major contributors,  
> it would be very very difficult, especially for less
> experienced  
> "volunteers", to maintain the release branches and
> fix the bugs.  
>  From my personal experience trying to create the
> opentaps releases,  
> a good release can only be created if the original
> version is  
> reasonably stable and if the core developers
> significantly support  
> the effort by helping to push the bug fixes from
> trunk to the release  
> branch.
>
> On the issue of stability:
>
> 1.  I propose that we put this
> http://jira.undersunconsulting.com/ 
> browse/OFBIZ-500 back into the main code base.  It
> addressed a  
> typecast issue with field-to-field.
>
> 2.  I think we should take a vote: how many people
> would like to keep  
> current code "as is", so the "OID" data type (used
> for storing images  
> and content) works with Derby and not PostgreSQL,
> versus making a  
> change which would make it work with PostgreSQL and
> not Derby?
>
> 3.  I'll just keep my fingers crossed about the
> Geronimo transactions  
> manager then.
>
> Si
>
>
> On Jul 24, 2006, at 10:13 AM, David E Jones wrote:
>
> >
>
>

[BJ Freeman]

Basically we have the Framework and the application.
so why not have an RC for the framework first, then RC for each application.
I know the problem with applications is interdependence. I believe this
method will bring about a better design that will allow one application
to go thru its process without worrying about what the dependencies on
the other app will do.



Si Chen sent the following on 7/24/2006 11:33 AM:

> David,
>
> Without coordinating a feature freeze between the major contributors, it
> would be very very difficult, especially for less experienced
> "volunteers", to maintain the release branches and fix the bugs.  From
> my personal experience trying to create the opentaps releases, a good
> release can only be created if the original version is reasonably stable
> and if the core developers significantly support the effort by helping
> to push the bug fixes from trunk to the release branch.
>
> On the issue of stability:
>
> 1.  I propose that we put this
> http://jira.undersunconsulting.com/browse/OFBIZ-500 back into the main
> code base.  It addressed a typecast issue with field-to-field.
>
> 2.  I think we should take a vote: how many people would like to keep
> current code "as is", so the "OID" data type (used for storing images
> and content) works with Derby and not PostgreSQL, versus making a change
> which would make it work with PostgreSQL and not Derby?
>
> 3.  I'll just keep my fingers crossed about the Geronimo transactions
> manager then.
>
> Si
>
>
> On Jul 24, 2006, at 10:13 AM, David E Jones wrote:
>
>>
>
>

[David E Jones-2]

Chris,

Exactly. If anyone knows of any I'd love to see how they're doing things... it's not an easy course in general, and I'm not sure if there are really many community driven enterprise level projects.

-David


Chris Howe wrote:

> Are there any open source projects that are not driven
> by a single company that successfully implement
> feature freeze releases AND would have the complexity
> of feature advancements that OFBiz does (this would
> exclude most, if not all, Apache projects as most of
> them are one trick ponies, so to speak)?  If there
> are, maybe we should see how they best accopmlish
> this.
>
> --- Si Chen <[hidden email]> wrote:
>
>> David,
>>
>> Without coordinating a feature freeze between the
>> major contributors,  
>> it would be very very difficult, especially for less
>> experienced  
>> "volunteers", to maintain the release branches and
>> fix the bugs.  
>>  From my personal experience trying to create the
>> opentaps releases,  
>> a good release can only be created if the original
>> version is  
>> reasonably stable and if the core developers
>> significantly support  
>> the effort by helping to push the bug fixes from
>> trunk to the release  
>> branch.
>>
>> On the issue of stability:
>>
>> 1.  I propose that we put this
>> http://jira.undersunconsulting.com/ 
>> browse/OFBIZ-500 back into the main code base.  It
>> addressed a  
>> typecast issue with field-to-field.
>>
>> 2.  I think we should take a vote: how many people
>> would like to keep  
>> current code "as is", so the "OID" data type (used
>> for storing images  
>> and content) works with Derby and not PostgreSQL,
>> versus making a  
>> change which would make it work with PostgreSQL and
>> not Derby?
>>
>> 3.  I'll just keep my fingers crossed about the
>> Geronimo transactions  
>> manager then.
>>
>> Si
>>
>>
>> On Jul 24, 2006, at 10:13 AM, David E Jones wrote:
>>
>>
>

[Si Chen-2]

Postgresql is a good example.

On Jul 24, 2006, at 1:52 PM, David E Jones wrote:

>
> Chris,
>
> Exactly. If anyone knows of any I'd love to see how they're doing  
> things... it's not an easy course in general, and I'm not sure if  
> there are really many community driven enterprise level projects.
>
> -David
>
>
> Chris Howe wrote:
>> Are there any open source projects that are not driven
>> by a single company that successfully implement
>> feature freeze releases AND would have the complexity
>> of feature advancements that OFBiz does (this would
>> exclude most, if not all, Apache projects as most of
>> them are one trick ponies, so to speak)?  If there
>> are, maybe we should see how they best accopmlish
>> this.
>> --- Si Chen <[hidden email]> wrote:
>>> David,
>>>
>>> Without coordinating a feature freeze between the
>>> major contributors,  it would be very very difficult, especially  
>>> for less
>>> experienced  "volunteers", to maintain the release branches and
>>> fix the bugs.    From my personal experience trying to create the
>>> opentaps releases,  a good release can only be created if the  
>>> original
>>> version is  reasonably stable and if the core developers
>>> significantly support  the effort by helping to push the bug  
>>> fixes from
>>> trunk to the release  branch.
>>>
>>> On the issue of stability:
>>>
>>> 1.  I propose that we put this
>>> http://jira.undersunconsulting.com/ browse/OFBIZ-500 back into  
>>> the main code base.  It
>>> addressed a  typecast issue with field-to-field.
>>>
>>> 2.  I think we should take a vote: how many people
>>> would like to keep  current code "as is", so the "OID" data type  
>>> (used
>>> for storing images  and content) works with Derby and not  
>>> PostgreSQL,
>>> versus making a  change which would make it work with PostgreSQL and
>>> not Derby?
>>>
>>> 3.  I'll just keep my fingers crossed about the
>>> Geronimo transactions  manager then.
>>>
>>> Si
>>>
>>>
>>> On Jul 24, 2006, at 10:13 AM, David E Jones wrote:
>>>
>>>

[David E Jones-2]

Si Chen wrote:
> David,
>
> Without coordinating a feature freeze between the major contributors, it
> would be very very difficult, especially for less experienced
> "volunteers", to maintain the release branches and fix the bugs.  From
> my personal experience trying to create the opentaps releases, a good
> release can only be created if the original version is reasonably stable
> and if the core developers significantly support the effort by helping
> to push the bug fixes from trunk to the release branch.

So, your hope in doing a feature freeze is to get more people involved with bug fixing and maintenance of the releases?

It's an interesting thought, but I think it might back-fire more than help. In other words, I'm guessing it would have an effect of reducing contributions for advancement of new features without increasing contributions for bug fixing and maintenance.

In general we're constrained somewhat by the natural forces (ie contracts, employers, free-time + interest, etc) that drive things into the project. Based on that my thoughts are centered around facilitating contribution and I believe that if we do a good job of this then it will drive new stuff for improvements as well as fixes.

> On the issue of stability:
>
> 1.  I propose that we put this
> http://jira.undersunconsulting.com/browse/OFBIZ-500 back into the main
> code base.  It addressed a typecast issue with field-to-field.

I'll try to take a look at this sometime this week. With any luck there will be some slow time at the show... ;)

> 2.  I think we should take a vote: how many people would like to keep
> current code "as is", so the "OID" data type (used for storing images
> and content) works with Derby and not PostgreSQL, versus making a change
> which would make it work with PostgreSQL and not Derby?

I'm having a hard time seeing a reason why we have to vote for supporting one and breaking the other.

At worst we should have a configuration parameter (like an attribute on the datasource element in the entityengine.xml file).

At best we should look into a REAL solution that works well for both and may address other current or potential issues with other databases.

My opinion right now is that whatever the case we should not break Derby to make things easier for Postgres... The OOTB settings use Derby and that should work properly as the test bed as well as for demoing.

I'll take a look at this if I can, but proper testing and playing with JDBC code changes would require at least half a day for me to set things up and play around and expect to dig deep enough to find a good solution. I can't commit to that for at least the next 2-3 weeks.

> 3.  I'll just keep my fingers crossed about the Geronimo transactions
> manager then.

Yeah, until we find more information (ie good test cases for problems or a good error message or something) there isn't much we can do...

-David

[cjhowe]

Is that really a comparable project?  I know next to
nothing about Postgresql, but their change log only
lists 13 changes over a 3 month time period between 7
people, all of which I would imagine have umpteen
years of experience in just database coding.

In any case, they're maintaining 4 versions (7.2, 7.3,
7.4, 8.0).

I don't think that OFBiz's structure (directory, file
sturcture, etc) is set in stone enough for maintained
release points to be of very much use to anyone
outside of marketing endeavors.  Everytime OFBiz
deletes a file because of reuse, it becomes impossible
to maintain any release prior to that delete (OFBiz
SVN will fix one bug, the release points would have to
find both bugs).

Just my .02

--- Si Chen <[hidden email]> wrote:

> Postgresql is a good example.
>
> On Jul 24, 2006, at 1:52 PM, David E Jones wrote:
>
> >
> > Chris,
> >
> > Exactly. If anyone knows of any I'd love to see
> how they're doing  
> > things... it's not an easy course in general, and
> I'm not sure if  
> > there are really many community driven enterprise
> level projects.
> >
> > -David
> >
> >
> > Chris Howe wrote:
> >> Are there any open source projects that are not
> driven
> >> by a single company that successfully implement
> >> feature freeze releases AND would have the
> complexity
> >> of feature advancements that OFBiz does (this
> would
> >> exclude most, if not all, Apache projects as most
> of
> >> them are one trick ponies, so to speak)?  If
> there
> >> are, maybe we should see how they best accopmlish
> >> this.
> >> --- Si Chen <[hidden email]>
> wrote:
> >>> David,
> >>>
> >>> Without coordinating a feature freeze between
> the
> >>> major contributors,  it would be very very
> difficult, especially  
> >>> for less
> >>> experienced  "volunteers", to maintain the
> release branches and
> >>> fix the bugs.    From my personal experience
> trying to create the
> >>> opentaps releases,  a good release can only be
> created if the  
> >>> original
> >>> version is  reasonably stable and if the core
> developers
> >>> significantly support  the effort by helping to
> push the bug  
> >>> fixes from
> >>> trunk to the release  branch.
> >>>
> >>> On the issue of stability:
> >>>
> >>> 1.  I propose that we put this
> >>> http://jira.undersunconsulting.com/
> browse/OFBIZ-500 back into  
> >>> the main code base.  It
> >>> addressed a  typecast issue with field-to-field.
> >>>
> >>> 2.  I think we should take a vote: how many
> people
> >>> would like to keep  current code "as is", so the
> "OID" data type  
> >>> (used
> >>> for storing images  and content) works with
> Derby and not  
> >>> PostgreSQL,
> >>> versus making a  change which would make it work
> with PostgreSQL and
> >>> not Derby?
> >>>
> >>> 3.  I'll just keep my fingers crossed about the
> >>> Geronimo transactions  manager then.
> >>>
> >>> Si
> >>>
> >>>
> >>> On Jul 24, 2006, at 10:13 AM, David E Jones
> wrote:
> >>>
> >>>
>
>

[David E Jones-2]

I guess I should have been more specific, I meant more along the lines of community driven business application projects and not the more general category of "enterprise level projects" as I wrote.

Postgres is an interesting example of a community driven project that is pretty independent. On the infrastructure level there are quite a few projects, like many under the Apache umbrella, that would be similar I guess.

The trick with business applications is that there is no industry standard to implement to, in other words there is no spec or standard from W3C, OMG, ANSI, Sun/JCP, etc that says "this is what constitutes an implementation of the GSBPBMS 1.0 (Global Standard Best Practices Business Management System) specification. It sure would be nice if such a thing existed....

Early on with OFBiz I researched a lot of standards from various groups like OMG, OAGIS, OASIS, and so on that would not define an entire system, but at least parts of a such a system. The UBL standard might be worth supporting, but there is the issue that not only might it be hard to find sponsors or volunteers for building it, but it might be hard to find users (beta or production) for it...

BTW, another interesting thing that Postgres and various ASF projects have is an origin as a product developed or owned by a company (either open source or commercial) that later became community-driven, or in other words a more publicity friendly alternative to abandonment of the software. ;) In a way it's unfortunate that OFBiz did not benefit from that sort of past as we might have been able to avoid some of the significant growing pains that were part of the more evolutionary process that brought the project (especially the framework, but very much also the applications) to where it is now.

-David


Si Chen wrote:

> Postgresql is a good example.
>
> On Jul 24, 2006, at 1:52 PM, David E Jones wrote:
>
>>
>> Chris,
>>
>> Exactly. If anyone knows of any I'd love to see how they're doing
>> things... it's not an easy course in general, and I'm not sure if
>> there are really many community driven enterprise level projects.
>>
>> -David
>>
>>
>> Chris Howe wrote:
>>> Are there any open source projects that are not driven
>>> by a single company that successfully implement
>>> feature freeze releases AND would have the complexity
>>> of feature advancements that OFBiz does (this would
>>> exclude most, if not all, Apache projects as most of
>>> them are one trick ponies, so to speak)?  If there
>>> are, maybe we should see how they best accopmlish
>>> this.
>>> --- Si Chen <[hidden email]> wrote:
>>>> David,
>>>>
>>>> Without coordinating a feature freeze between the
>>>> major contributors,  it would be very very difficult, especially for
>>>> less
>>>> experienced  "volunteers", to maintain the release branches and
>>>> fix the bugs.    From my personal experience trying to create the
>>>> opentaps releases,  a good release can only be created if the original
>>>> version is  reasonably stable and if the core developers
>>>> significantly support  the effort by helping to push the bug fixes from
>>>> trunk to the release  branch.
>>>>
>>>> On the issue of stability:
>>>>
>>>> 1.  I propose that we put this
>>>> http://jira.undersunconsulting.com/ browse/OFBIZ-500 back into the
>>>> main code base.  It
>>>> addressed a  typecast issue with field-to-field.
>>>>
>>>> 2.  I think we should take a vote: how many people
>>>> would like to keep  current code "as is", so the "OID" data type (used
>>>> for storing images  and content) works with Derby and not PostgreSQL,
>>>> versus making a  change which would make it work with PostgreSQL and
>>>> not Derby?
>>>>
>>>> 3.  I'll just keep my fingers crossed about the
>>>> Geronimo transactions  manager then.
>>>>
>>>> Si
>>>>
>>>>
>>>> On Jul 24, 2006, at 10:13 AM, David E Jones wrote:
>>>>
>>>>
>

[Ruth Hoffman]

David .et .al:

re: Geronimo "issues"
I still have an unresolved "issue" reading a database. Have no idea if
it's Geronimo, Postgres or something in between or beyond. Granted, it
is a big DB with over 14Million records, but Webtools Entity Reference &
XML exports don't work. I run out of memory, regardless of how much
memory I allocate during system startup. I do know that simple Postgres
psql and pgAdmin can read this database and return from a simple "select
*" query of the (single) table in question.

I agree that this isn't a show stopper and does not need to be resolved
for the release, but...is there even a next step in tracking down issues
such as this? Is there anything else I can do to help isolate and
perhaps correct this problem?

Ruth

David E Jones wrote:

>
>
> Si Chen wrote:
>
>> Hi everybody -
>>
>> I'm really glad we will be doing a release for OFBIZ again.  Here are
>> a things I would really like to see -
>
>
> [snip]
>
>> 3.  Separately, there are some production-readiness issues which we
>> should address prior to a release.  For example, there was a
>> typecasting issue for the minilang field-to-field and a PostgreSQL
>> bytea/oid issue that conflicted with Derby which were never
>> satisfactorily resolved.  I think if we're going to put out a release
>> and tell our users "this is ready for production use", we should
>> address those.
>>
>> 4.  Also, has the new Geronimo transaction manager been fully
>> tested?  Are the reports of transaction time out issues real?
>
>
> With the branch process I mentioned in the other email these issues
> don't matter so much.
>
> Yes, we want to have something as close to production-ready as
> possible, but OFBiz is a big system that supports a really wide
> variety of lower lever software to interact with, so there is no way
> we can guarantee (unless someone has a bunch of money socked away that
> they want to dedicate to a testing lab...) that OFBiz will work OOTB
> with all combinations of infrastructure software without subtle
> configuration and/or code changes specific to a circumstance.
>
> So, while these two issues are worth keeping an eye on and fixing when
> we can, I don't think either is a show-stopper for a release, even a
> "production stable" release.
>
> For both of these it may be that those who are able to fix such things
> don't have the resources to do so now, but any reasonable size
> deployment team usually has people around who can. Also, such things
> are often easier to fix for a specific circumstance than something
> that is meant to work in a wide variety of cases, which is the case
> with the Postgres issue. With a little creativity and experience it is
> possible to fix in a very widely applicable way any such issue and not
> have any problems with Derby or Postgres, but with no resources to do
> so... the best we can do is get a release out and attract more
> resources to the community...
>
> On the Geronimo issue: I still haven't seen reports of anything that
> seems to be a core issue with Geronimo. People report errors all the
> time that are caused by things like timeouts and such or a lower level
> error that causes rollback-only to be set and these manifest higher up
> as transaction errors, but the details are in the logs. So far I
> haven't seen any clear definition of a problem with Geronimo. That
> doesn't mean there isn't one, it just means that from what I've seen
> on the lists it hasn't been isolated yet (ie seen by someone who can
> recognize it for what it is and distinguish between a real Geronimo
> problem and a simple timeout or other error that caused the
> transaction error).
>
> -David
>
>

[davidnwelton]

> Are there any open source projects that are not driven
> by a single company that successfully implement
> feature freeze releases AND would have the complexity
> of feature advancements that OFBiz does (this would
> exclude most, if not all, Apache projects as most of
> them are one trick ponies, so to speak)?  If there
> are, maybe we should see how they best accopmlish
> this.

OFBiz is complex, but I don't think it's *that* much of an outlier.
The Linux kernel has a lot of complex subsystems that need to
interact.  Going one step further, 'distribution' type projects
(FreeBSD, that includes the kernel and user space, or something like
Debian or Ubuntu that concentrates on aggregating the pieces), have to
contend with constantly changing pieces of the system, and attempting
to freeze a stable set.  Different from OFBiz, but I think it's not an
impossible problem.  Personally, I think a huge win would be to
increase the amount of automated regression testing, but that's
certainly an effort.

I think it's partially an issue of time allocation - it seems that
currently, most time is spent 'going forward' rather than solidifying,
testing, and so on.  I have the utmost confidence that were the guys
behind OFBiz to change their focus some, they would do the same
excellent job in cutting stable releases.  But it's their call...

--
David N. Welton
 - http://www.dedasys.com/davidw/

Linux, Open Source Consulting
 - http://www.dedasys.com/

[BJ Freeman]

In a recent email david mentioned that he re-wrote the webtools to
export from a db using cursors so a lot of memory was not needed.

You might check if your version of ofbiz has the change. Also if your db
supports cursors.


Ruth Hoffman sent the following on 7/25/2006 7:47 AM:

> David .et .al:
>
> re: Geronimo "issues"
> I still have an unresolved "issue" reading a database. Have no idea if
> it's Geronimo, Postgres or something in between or beyond. Granted, it
> is a big DB with over 14Million records, but Webtools Entity Reference &
> XML exports don't work. I run out of memory, regardless of how much
> memory I allocate during system startup. I do know that simple Postgres
> psql and pgAdmin can read this database and return from a simple "select
> *" query of the (single) table in question.
>
> I agree that this isn't a show stopper and does not need to be resolved
> for the release, but...is there even a next step in tracking down issues
> such as this? Is there anything else I can do to help isolate and
> perhaps correct this problem?
>
> Ruth
>
> David E Jones wrote:
>
>>
>>
>> Si Chen wrote:
>>
>>> Hi everybody -
>>>
>>> I'm really glad we will be doing a release for OFBIZ again.  Here are
>>> a things I would really like to see -
>>
>>
>> [snip]
>>
>>> 3.  Separately, there are some production-readiness issues which we
>>> should address prior to a release.  For example, there was a
>>> typecasting issue for the minilang field-to-field and a PostgreSQL
>>> bytea/oid issue that conflicted with Derby which were never
>>> satisfactorily resolved.  I think if we're going to put out a release
>>> and tell our users "this is ready for production use", we should
>>> address those.
>>>
>>> 4.  Also, has the new Geronimo transaction manager been fully
>>> tested?  Are the reports of transaction time out issues real?
>>
>>
>> With the branch process I mentioned in the other email these issues
>> don't matter so much.
>>
>> Yes, we want to have something as close to production-ready as
>> possible, but OFBiz is a big system that supports a really wide
>> variety of lower lever software to interact with, so there is no way
>> we can guarantee (unless someone has a bunch of money socked away that
>> they want to dedicate to a testing lab...) that OFBiz will work OOTB
>> with all combinations of infrastructure software without subtle
>> configuration and/or code changes specific to a circumstance.
>>
>> So, while these two issues are worth keeping an eye on and fixing when
>> we can, I don't think either is a show-stopper for a release, even a
>> "production stable" release.
>>
>> For both of these it may be that those who are able to fix such things
>> don't have the resources to do so now, but any reasonable size
>> deployment team usually has people around who can. Also, such things
>> are often easier to fix for a specific circumstance than something
>> that is meant to work in a wide variety of cases, which is the case
>> with the Postgres issue. With a little creativity and experience it is
>> possible to fix in a very widely applicable way any such issue and not
>> have any problems with Derby or Postgres, but with no resources to do
>> so... the best we can do is get a release out and attract more
>> resources to the community...
>>
>> On the Geronimo issue: I still haven't seen reports of anything that
>> seems to be a core issue with Geronimo. People report errors all the
>> time that are caused by things like timeouts and such or a lower level
>> error that causes rollback-only to be set and these manifest higher up
>> as transaction errors, but the details are in the logs. So far I
>> haven't seen any clear definition of a problem with Geronimo. That
>> doesn't mean there isn't one, it just means that from what I've seen
>> on the lists it hasn't been isolated yet (ie seen by someone who can
>> recognize it for what it is and distinguish between a real Geronimo
>> problem and a simple timeout or other error that caused the
>> transaction error).
>>
>> -David
>>
>>
>

[Ruth Hoffman]

Thanks BJ
will do
Ruth

BJ Freeman wrote:

> In a recent email david mentioned that he re-wrote the webtools to
> export from a db using cursors so a lot of memory was not needed.
>
> You might check if your version of ofbiz has the change. Also if your
> db supports cursors.
>
>
> Ruth Hoffman sent the following on 7/25/2006 7:47 AM:
>
>> David .et .al:
>>
>> re: Geronimo "issues"
>> I still have an unresolved "issue" reading a database. Have no idea
>> if it's Geronimo, Postgres or something in between or beyond.
>> Granted, it is a big DB with over 14Million records, but Webtools
>> Entity Reference & XML exports don't work. I run out of memory,
>> regardless of how much memory I allocate during system startup. I do
>> know that simple Postgres psql and pgAdmin can read this database and
>> return from a simple "select *" query of the (single) table in question.
>>
>> I agree that this isn't a show stopper and does not need to be
>> resolved for the release, but...is there even a next step in tracking
>> down issues such as this? Is there anything else I can do to help
>> isolate and perhaps correct this problem?
>>
>> Ruth
>>
>> David E Jones wrote:
>>
>>>
>>>
>>> Si Chen wrote:
>>>
>>>> Hi everybody -
>>>>
>>>> I'm really glad we will be doing a release for OFBIZ again.  Here
>>>> are a things I would really like to see -
>>>
>>>
>>>
>>> [snip]
>>>
>>>> 3.  Separately, there are some production-readiness issues which we
>>>> should address prior to a release.  For example, there was a
>>>> typecasting issue for the minilang field-to-field and a PostgreSQL
>>>> bytea/oid issue that conflicted with Derby which were never
>>>> satisfactorily resolved.  I think if we're going to put out a
>>>> release and tell our users "this is ready for production use", we
>>>> should address those.
>>>>
>>>> 4.  Also, has the new Geronimo transaction manager been fully
>>>> tested?  Are the reports of transaction time out issues real?
>>>
>>>
>>>
>>> With the branch process I mentioned in the other email these issues
>>> don't matter so much.
>>>
>>> Yes, we want to have something as close to production-ready as
>>> possible, but OFBiz is a big system that supports a really wide
>>> variety of lower lever software to interact with, so there is no way
>>> we can guarantee (unless someone has a bunch of money socked away
>>> that they want to dedicate to a testing lab...) that OFBiz will work
>>> OOTB with all combinations of infrastructure software without subtle
>>> configuration and/or code changes specific to a circumstance.
>>>
>>> So, while these two issues are worth keeping an eye on and fixing
>>> when we can, I don't think either is a show-stopper for a release,
>>> even a "production stable" release.
>>>
>>> For both of these it may be that those who are able to fix such
>>> things don't have the resources to do so now, but any reasonable
>>> size deployment team usually has people around who can. Also, such
>>> things are often easier to fix for a specific circumstance than
>>> something that is meant to work in a wide variety of cases, which is
>>> the case with the Postgres issue. With a little creativity and
>>> experience it is possible to fix in a very widely applicable way any
>>> such issue and not have any problems with Derby or Postgres, but
>>> with no resources to do so... the best we can do is get a release
>>> out and attract more resources to the community...
>>>
>>> On the Geronimo issue: I still haven't seen reports of anything that
>>> seems to be a core issue with Geronimo. People report errors all the
>>> time that are caused by things like timeouts and such or a lower
>>> level error that causes rollback-only to be set and these manifest
>>> higher up as transaction errors, but the details are in the logs. So
>>> far I haven't seen any clear definition of a problem with Geronimo.
>>> That doesn't mean there isn't one, it just means that from what I've
>>> seen on the lists it hasn't been isolated yet (ie seen by someone
>>> who can recognize it for what it is and distinguish between a real
>>> Geronimo problem and a simple timeout or other error that caused the
>>> transaction error).
>>>
>>> -David
>>>
>>>
>>
>

[David E Jones-2]

On Jul 24, 2006, at 12:33 PM, Si Chen wrote:

> On the issue of stability:
>
> 1.  I propose that we put this http://jira.undersunconsulting.com/ 
> browse/OFBIZ-500 back into the main code base.  It addressed a  
> typecast issue with field-to-field.

I just took a look at this issue and it seems to be closed and  
acknowledged as resolved. Is there something I'm missing?

-David

[Si Chen-2]

The particular issue is marked as resolved by re-writing the service  
in question, but the patch for field-to-field typecasting was never  
put back into minilang, so other services would break.

Si


On Jul 27, 2006, at 1:29 PM, David E Jones wrote:

>
> On Jul 24, 2006, at 12:33 PM, Si Chen wrote:
>
>> On the issue of stability:
>>
>> 1.  I propose that we put this http://jira.undersunconsulting.com/ 
>> browse/OFBIZ-500 back into the main code base.  It addressed a  
>> typecast issue with field-to-field.
>
> I just took a look at this issue and it seems to be closed and  
> acknowledged as resolved. Is there something I'm missing?
>
> -David

[David E Jones-2]

Okay, I think I see what you are saying. I'm not sure if I like the  
change though... If we do it for field-to-field we would really need  
to do it for field-to-env, env-to-field, and env-to-env. My hope for  
those fields is to get them totally removed from the simple-method  
language. They are already "deprecated" and you'll get log warnings  
if you use them. They are really old and stem from the days before  
the FlexibleStringExpander and FlexibleMapAccessor were implemented.

I may be missing something though... If there is a reason I'm missing  
why we really can't let this dead dog lie, please let me know.

-David


On Jul 27, 2006, at 2:35 PM, Si Chen wrote:

> The particular issue is marked as resolved by re-writing the  
> service in question, but the patch for field-to-field typecasting  
> was never put back into minilang, so other services would break.
>
> Si
>
>
> On Jul 27, 2006, at 1:29 PM, David E Jones wrote:
>
>>
>> On Jul 24, 2006, at 12:33 PM, Si Chen wrote:
>>
>>> On the issue of stability:
>>>
>>> 1.  I propose that we put this http://jira.undersunconsulting.com/ 
>>> browse/OFBIZ-500 back into the main code base.  It addressed a  
>>> typecast issue with field-to-field.
>>
>> I just took a look at this issue and it seems to be closed and  
>> acknowledged as resolved. Is there something I'm missing?
>>
>> -David
>

[Jacques Le Roux]

From: "David E Jones" <[hidden email]>


>
> Okay, I think I see what you are saying. I'm not sure if I like the
> change though... If we do it for field-to-field we would really need
> to do it for field-to-env, env-to-field, and env-to-env. My hope for
> those fields is to get them totally removed from the simple-method
> language. They are already "deprecated" and you'll get log warnings
> if you use them. They are really old and stem from the days before
> the FlexibleStringExpander and FlexibleMapAccessor were implemented.

In this spirit it may me good that everybody seing these deprecated warning change these calls for set calls. This will clear the
log a littlle bit

TIA

Jacques

> I may be missing something though... If there is a reason I'm missing
> why we really can't let this dead dog lie, please let me know.
>
> -David
>
>
> On Jul 27, 2006, at 2:35 PM, Si Chen wrote:
>
> > The particular issue is marked as resolved by re-writing the
> > service in question, but the patch for field-to-field typecasting
> > was never put back into minilang, so other services would break.
> >
> > Si
> >
> >
> > On Jul 27, 2006, at 1:29 PM, David E Jones wrote:
> >
> >>
> >> On Jul 24, 2006, at 12:33 PM, Si Chen wrote:
> >>
> >>> On the issue of stability:
> >>>
> >>> 1.  I propose that we put this http://jira.undersunconsulting.com/
> >>> browse/OFBIZ-500 back into the main code base.  It addressed a
> >>> typecast issue with field-to-field.
> >>
> >> I just took a look at this issue and it seems to be closed and
> >> acknowledged as resolved. Is there something I'm missing?
> >>
> >> -David
> >

[David E Jones-2]

On Jul 28, 2006, at 12:28 AM, Jacques Le Roux wrote:

>> Okay, I think I see what you are saying. I'm not sure if I like the
>> change though... If we do it for field-to-field we would really need
>> to do it for field-to-env, env-to-field, and env-to-env. My hope for
>> those fields is to get them totally removed from the simple-method
>> language. They are already "deprecated" and you'll get log warnings
>> if you use them. They are really old and stem from the days before
>> the FlexibleStringExpander and FlexibleMapAccessor were implemented.
>
> In this spirit it may me good that everybody seing these deprecated  
> warning change these calls for set calls. This will clear the
> log a littlle bit

Yes, definitely.

-David