Design improvements?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
28 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: Design improvements?

BJ Freeman
not sure where you coming from james
I have stated that all customization is done outside of the SVN image,
by either hot-deploy or a seperate folder(which is what I use) of
components the either override or access the svn image.

and that a automated way to check those customization against the
changes in the svn image to see if there are any changes necessary to
the customization would greatly reduce the cost of maintenance.

I have not said that any Customization should be in the svn image or
part of the svn.




=========================
BJ Freeman
Strategic Power Office with Supplier Automation  <http://www.businessesnetwork.com/automation/viewforum.php?f=52>
Specialtymarket.com  <http://www.specialtymarket.com/>
Systems Integrator-- Glad to Assist

Chat  Y! messenger: bjfr33man


james_sg sent the following on 9/22/2010 6:59 PM:

>
> Hi BJ,
>
> If we customize an application by configuration, upgrades are easier.
> If we customize by replacing the codes, upgrades will not be possible.
> This means functions / features in OFBiz should be able to turn on / off.
>
> Regards,
> James
>
>
> BJ Freeman wrote:
>>
>> I disagree. every complicated process is made up of simple processes.
>> the level you approach it, is what makes it complicated.
>> An I believe the process can be modeled and implemented.
>>
>> just as the tests are now, as limited as they are.
>>
>> what it takes is a commitment by each one that changes the code base to
>> put their smarts in the direction of what does it take to get from here
>> to there as far as migration.
>>
>> one way for instance is to use Diffs or even the patch itself as a basis
>> for driving the migration.
>>
>> then is it a matter of tools to use the diffs and patches to run a check
>> on customization.
>>
>> I do maintain my own SVN, I would not be this far if I did not.
>> however it does not help to compare code that outside that of the ofbiz
>> svn(component that are customized but not part of the code base of ofbiz)
>>
>> The bottom line, for a business, is overhead to maintain it hard cash.
>>
>>
>>
>>
>> Jonatan Soto sent the following on 9/22/2010 5:00 PM:
>>> What I see here is that it is not as easy to create an upgrade system for
>>> this kind of project. Perhaps the nature of it, it's a good reason.
>>> Remember
>>> that Ofbiz is an ERP system and tries to cover as much as it can
>>> different
>>> businesses, so IMO it's not like a proprietary product or another kind of
>>> open source project where it is not a common practice to change the trunk
>>> code by everyone and also allows to easily install plugins, mods,
>>> extensions, etc.
>>>
>>> IMO, I can say that one good chance to achieve this is to separate the
>>> framework from the apps, already discussed in a lot of posts and I think
>>> it
>>> is strongly accepted by the community. This will get the ability to
>>> extract
>>> an important piece of code which I think is not regularly altered, at
>>> least
>>> in my case, I've modified some apps but never the framework.
>>>>  From my point of view, the hot-deploy works fine and it's fairly enough
>>>> for
>>> a lot of customization purposes and maintain the trunk code untouched.
>>> But
>>> it seems to me that sometimes is not enough mostly when an adaptation of
>>> an
>>> existing app, concretely i18n enters the scene. For example I found to be
>>> very complicated to move the entire Accounting module, IMO the one that
>>> requires more i18n, since it is related with a lot of modules. In this
>>> case
>>> I have had to customize in-place. I have some ideas about that, but I
>>> prefer
>>> to discuss this in a proper post.
>>>
>>> BJ, I believe that your problem is related with the number of
>>> installations
>>> do you have/maintain so a little solution may be (if you aren't doing
>>> yet)
>>> the vendor drop technique (
>>> http://svnbook.red-bean.com/en/1.5/svn.advanced.vendorbr.html). But I
>>> suppose it's only valid if you decided to use the trunk version from the
>>> beginning and kept up to date so often...
>>>
>>> On Thu, Sep 23, 2010 at 12:44 AM, Jacques Le Roux<
>>> [hidden email]>   wrote:
>>>
>>>> From: "Adam Heath"<[hidden email]>
>>>>
>>>>    On 09/22/2010 03:33 PM, BJ Freeman wrote:
>>>>>
>>>>>> Am it to gather by this that you can do a direct SVN update and all
>>>>>> your
>>>>>> customization continue to work?
>>>>>> say from 9.04 to 10.04
>>>>>>
>>>>>
>>>>> Ofbiz has never supported upgrades.  I agree with BJ here.
>>>>>
>>>>> Database tables can change.  Not all changes are automatic.  Such
>>>>> changes
>>>>> are not listed in a simple place(in the source).
>>>>>
>>>>> Values in tables can change.  No upgrade conversions are
>>>>> provided(again,
>>>>> in a simple place).
>>>>>
>>>>
>>>> Is this useless?
>>>>
>>>> https://cwiki.apache.org/confluence/display/OFBTECH/Revisions+Requiring+Data+Migration
>>>>
>>>> Jacques
>>>>
>>>>
>>>>    To solve this, requires running older ofbiz on older database, doing a
>>>>> test upgrade to each and every new version, and then seeing what is
>>>>> different.  This is a *very* hard problem, not easy to automate, and
>>>>> takes
>>>>> smart people lots of time.  This is not something you can just force on
>>>>> the
>>>>> community.  No one has sat down to do this very busy, hard work, so it
>>>>> hasn't yet been done.
>>>>>
>>>>> If you run trunk, then it is up to you to solve any per-version upgrade
>>>>> problems.
>>>>>
>>>>> However, official releases should really have appropriate, detailed,
>>>>> upgrade instructions.
>>>>>
>>>>>
>>>>
>>>
>>>
>>
>>
>>
>

Reply | Threaded
Open this post in threaded view
|

Re: Design improvements?

james_sg
hi BJ,

To give an example.
Let say if i wanted to hide some menu items, i comment them out.
When i upgrade this implementation, i need to make sure the modification is not overriden by newer codes.
If i can hide the menu items just by changing some data in the database, future upgrades will be easier as less codes need to be checked.

Also applies to the screenlets. If I wanted to hide them, it is better to be able to made the change by modifying the database.

Regards,
James

BJ Freeman wrote
not sure where you coming from james
I have stated that all customization is done outside of the SVN image,
by either hot-deploy or a seperate folder(which is what I use) of
components the either override or access the svn image.

and that a automated way to check those customization against the
changes in the svn image to see if there are any changes necessary to
the customization would greatly reduce the cost of maintenance.

I have not said that any Customization should be in the svn image or
part of the svn.




=========================
BJ Freeman
Strategic Power Office with Supplier Automation  <http://www.businessesnetwork.com/automation/viewforum.php?f=52>
Specialtymarket.com  <http://www.specialtymarket.com/>
Systems Integrator-- Glad to Assist

Chat  Y! messenger: bjfr33man


james_sg sent the following on 9/22/2010 6:59 PM:
>
> Hi BJ,
>
> If we customize an application by configuration, upgrades are easier.
> If we customize by replacing the codes, upgrades will not be possible.
> This means functions / features in OFBiz should be able to turn on / off.
>
> Regards,
> James
>
>
> BJ Freeman wrote:
>>
>> I disagree. every complicated process is made up of simple processes.
>> the level you approach it, is what makes it complicated.
>> An I believe the process can be modeled and implemented.
>>
>> just as the tests are now, as limited as they are.
>>
>> what it takes is a commitment by each one that changes the code base to
>> put their smarts in the direction of what does it take to get from here
>> to there as far as migration.
>>
>> one way for instance is to use Diffs or even the patch itself as a basis
>> for driving the migration.
>>
>> then is it a matter of tools to use the diffs and patches to run a check
>> on customization.
>>
>> I do maintain my own SVN, I would not be this far if I did not.
>> however it does not help to compare code that outside that of the ofbiz
>> svn(component that are customized but not part of the code base of ofbiz)
>>
>> The bottom line, for a business, is overhead to maintain it hard cash.
>>
>>
>>
>>
>> Jonatan Soto sent the following on 9/22/2010 5:00 PM:
>>> What I see here is that it is not as easy to create an upgrade system for
>>> this kind of project. Perhaps the nature of it, it's a good reason.
>>> Remember
>>> that Ofbiz is an ERP system and tries to cover as much as it can
>>> different
>>> businesses, so IMO it's not like a proprietary product or another kind of
>>> open source project where it is not a common practice to change the trunk
>>> code by everyone and also allows to easily install plugins, mods,
>>> extensions, etc.
>>>
>>> IMO, I can say that one good chance to achieve this is to separate the
>>> framework from the apps, already discussed in a lot of posts and I think
>>> it
>>> is strongly accepted by the community. This will get the ability to
>>> extract
>>> an important piece of code which I think is not regularly altered, at
>>> least
>>> in my case, I've modified some apps but never the framework.
>>>>  From my point of view, the hot-deploy works fine and it's fairly enough
>>>> for
>>> a lot of customization purposes and maintain the trunk code untouched.
>>> But
>>> it seems to me that sometimes is not enough mostly when an adaptation of
>>> an
>>> existing app, concretely i18n enters the scene. For example I found to be
>>> very complicated to move the entire Accounting module, IMO the one that
>>> requires more i18n, since it is related with a lot of modules. In this
>>> case
>>> I have had to customize in-place. I have some ideas about that, but I
>>> prefer
>>> to discuss this in a proper post.
>>>
>>> BJ, I believe that your problem is related with the number of
>>> installations
>>> do you have/maintain so a little solution may be (if you aren't doing
>>> yet)
>>> the vendor drop technique (
>>> http://svnbook.red-bean.com/en/1.5/svn.advanced.vendorbr.html). But I
>>> suppose it's only valid if you decided to use the trunk version from the
>>> beginning and kept up to date so often...
>>>
>>> On Thu, Sep 23, 2010 at 12:44 AM, Jacques Le Roux<
>>> jacques.le.roux@les7arts.com>   wrote:
>>>
>>>> From: "Adam Heath"<doogie@brainfood.com>
>>>>
>>>>    On 09/22/2010 03:33 PM, BJ Freeman wrote:
>>>>>
>>>>>> Am it to gather by this that you can do a direct SVN update and all
>>>>>> your
>>>>>> customization continue to work?
>>>>>> say from 9.04 to 10.04
>>>>>>
>>>>>
>>>>> Ofbiz has never supported upgrades.  I agree with BJ here.
>>>>>
>>>>> Database tables can change.  Not all changes are automatic.  Such
>>>>> changes
>>>>> are not listed in a simple place(in the source).
>>>>>
>>>>> Values in tables can change.  No upgrade conversions are
>>>>> provided(again,
>>>>> in a simple place).
>>>>>
>>>>
>>>> Is this useless?
>>>>
>>>> https://cwiki.apache.org/confluence/display/OFBTECH/Revisions+Requiring+Data+Migration
>>>>
>>>> Jacques
>>>>
>>>>
>>>>    To solve this, requires running older ofbiz on older database, doing a
>>>>> test upgrade to each and every new version, and then seeing what is
>>>>> different.  This is a *very* hard problem, not easy to automate, and
>>>>> takes
>>>>> smart people lots of time.  This is not something you can just force on
>>>>> the
>>>>> community.  No one has sat down to do this very busy, hard work, so it
>>>>> hasn't yet been done.
>>>>>
>>>>> If you run trunk, then it is up to you to solve any per-version upgrade
>>>>> problems.
>>>>>
>>>>> However, official releases should really have appropriate, detailed,
>>>>> upgrade instructions.
>>>>>
>>>>>
>>>>
>>>
>>>
>>
>>
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: Design improvements?

BJ Freeman
Yes I don't cover that type of customization,
I copy the component into hot-deploy for simplicity.
then I modify the component I copied.
when an update is done from the svn the original component may get updated.

now I what to know if what I did in my copied component was effected.
Like they removed the menu I commented out, or they added screenlets.

the time it take to evaluate then is what I am focusing on automating.

james_sg sent the following on 9/22/2010 8:43 PM:

>
> hi BJ,
>
> To give an example.
> Let say if i wanted to hide some menu items, i comment them out.
> When i upgrade this implementation, i need to make sure the modification is
> not overriden by newer codes.
> If i can hide the menu items just by changing some data in the database,
> future upgrades will be easier as less codes need to be checked.
>
> Also applies to the screenlets. If I wanted to hide them, it is better to be
> able to made the change by modifying the database.
>
> Regards,
> James
>
>
> BJ Freeman wrote:
>>
>> not sure where you coming from james
>> I have stated that all customization is done outside of the SVN image,
>> by either hot-deploy or a seperate folder(which is what I use) of
>> components the either override or access the svn image.
>>
>> and that a automated way to check those customization against the
>> changes in the svn image to see if there are any changes necessary to
>> the customization would greatly reduce the cost of maintenance.
>>
>> I have not said that any Customization should be in the svn image or
>> part of the svn.
>>
>>
>>
>>
>> =========================
>> BJ Freeman
>> Strategic Power Office with Supplier Automation
>> <http://www.businessesnetwork.com/automation/viewforum.php?f=52>
>> Specialtymarket.com<http://www.specialtymarket.com/>
>> Systems Integrator-- Glad to Assist
>>
>> Chat  Y! messenger: bjfr33man
>>
>>
>> james_sg sent the following on 9/22/2010 6:59 PM:
>>>
>>> Hi BJ,
>>>
>>> If we customize an application by configuration, upgrades are easier.
>>> If we customize by replacing the codes, upgrades will not be possible.
>>> This means functions / features in OFBiz should be able to turn on / off.
>>>
>>> Regards,
>>> James
>>>
>>>
>>> BJ Freeman wrote:
>>>>
>>>> I disagree. every complicated process is made up of simple processes.
>>>> the level you approach it, is what makes it complicated.
>>>> An I believe the process can be modeled and implemented.
>>>>
>>>> just as the tests are now, as limited as they are.
>>>>
>>>> what it takes is a commitment by each one that changes the code base to
>>>> put their smarts in the direction of what does it take to get from here
>>>> to there as far as migration.
>>>>
>>>> one way for instance is to use Diffs or even the patch itself as a basis
>>>> for driving the migration.
>>>>
>>>> then is it a matter of tools to use the diffs and patches to run a check
>>>> on customization.
>>>>
>>>> I do maintain my own SVN, I would not be this far if I did not.
>>>> however it does not help to compare code that outside that of the ofbiz
>>>> svn(component that are customized but not part of the code base of
>>>> ofbiz)
>>>>
>>>> The bottom line, for a business, is overhead to maintain it hard cash.
>>>>
>>>>
>>>>
>>>>
>>>> Jonatan Soto sent the following on 9/22/2010 5:00 PM:
>>>>> What I see here is that it is not as easy to create an upgrade system
>>>>> for
>>>>> this kind of project. Perhaps the nature of it, it's a good reason.
>>>>> Remember
>>>>> that Ofbiz is an ERP system and tries to cover as much as it can
>>>>> different
>>>>> businesses, so IMO it's not like a proprietary product or another kind
>>>>> of
>>>>> open source project where it is not a common practice to change the
>>>>> trunk
>>>>> code by everyone and also allows to easily install plugins, mods,
>>>>> extensions, etc.
>>>>>
>>>>> IMO, I can say that one good chance to achieve this is to separate the
>>>>> framework from the apps, already discussed in a lot of posts and I
>>>>> think
>>>>> it
>>>>> is strongly accepted by the community. This will get the ability to
>>>>> extract
>>>>> an important piece of code which I think is not regularly altered, at
>>>>> least
>>>>> in my case, I've modified some apps but never the framework.
>>>>>>    From my point of view, the hot-deploy works fine and it's fairly
>>>>>> enough
>>>>>> for
>>>>> a lot of customization purposes and maintain the trunk code untouched.
>>>>> But
>>>>> it seems to me that sometimes is not enough mostly when an adaptation
>>>>> of
>>>>> an
>>>>> existing app, concretely i18n enters the scene. For example I found to
>>>>> be
>>>>> very complicated to move the entire Accounting module, IMO the one that
>>>>> requires more i18n, since it is related with a lot of modules. In this
>>>>> case
>>>>> I have had to customize in-place. I have some ideas about that, but I
>>>>> prefer
>>>>> to discuss this in a proper post.
>>>>>
>>>>> BJ, I believe that your problem is related with the number of
>>>>> installations
>>>>> do you have/maintain so a little solution may be (if you aren't doing
>>>>> yet)
>>>>> the vendor drop technique (
>>>>> http://svnbook.red-bean.com/en/1.5/svn.advanced.vendorbr.html). But I
>>>>> suppose it's only valid if you decided to use the trunk version from
>>>>> the
>>>>> beginning and kept up to date so often...
>>>>>
>>>>> On Thu, Sep 23, 2010 at 12:44 AM, Jacques Le Roux<
>>>>> [hidden email]>    wrote:
>>>>>
>>>>>> From: "Adam Heath"<[hidden email]>
>>>>>>
>>>>>>     On 09/22/2010 03:33 PM, BJ Freeman wrote:
>>>>>>>
>>>>>>>> Am it to gather by this that you can do a direct SVN update and all
>>>>>>>> your
>>>>>>>> customization continue to work?
>>>>>>>> say from 9.04 to 10.04
>>>>>>>>
>>>>>>>
>>>>>>> Ofbiz has never supported upgrades.  I agree with BJ here.
>>>>>>>
>>>>>>> Database tables can change.  Not all changes are automatic.  Such
>>>>>>> changes
>>>>>>> are not listed in a simple place(in the source).
>>>>>>>
>>>>>>> Values in tables can change.  No upgrade conversions are
>>>>>>> provided(again,
>>>>>>> in a simple place).
>>>>>>>
>>>>>>
>>>>>> Is this useless?
>>>>>>
>>>>>> https://cwiki.apache.org/confluence/display/OFBTECH/Revisions+Requiring+Data+Migration
>>>>>>
>>>>>> Jacques
>>>>>>
>>>>>>
>>>>>>     To solve this, requires running older ofbiz on older database,
>>>>>> doing a
>>>>>>> test upgrade to each and every new version, and then seeing what is
>>>>>>> different.  This is a *very* hard problem, not easy to automate, and
>>>>>>> takes
>>>>>>> smart people lots of time.  This is not something you can just force
>>>>>>> on
>>>>>>> the
>>>>>>> community.  No one has sat down to do this very busy, hard work, so
>>>>>>> it
>>>>>>> hasn't yet been done.
>>>>>>>
>>>>>>> If you run trunk, then it is up to you to solve any per-version
>>>>>>> upgrade
>>>>>>> problems.
>>>>>>>
>>>>>>> However, official releases should really have appropriate, detailed,
>>>>>>> upgrade instructions.
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>
>>
>>
>

Reply | Threaded
Open this post in threaded view
|

Re: Design improvements?

David E. Jones-2
In reply to this post by Adam Heath-2

On Sep 22, 2010, at 2:38 PM, Adam Heath wrote:

> On 09/22/2010 03:33 PM, BJ Freeman wrote:
>> Am it to gather by this that you can do a direct SVN update and all your
>> customization continue to work?
>> say from 9.04 to 10.04
>
> Ofbiz has never supported upgrades.  I agree with BJ here.
>
> Database tables can change.  Not all changes are automatic.  Such changes are not listed in a simple place(in the source).
>
> Values in tables can change.  No upgrade conversions are provided(again, in a simple place).
>
> To solve this, requires running older ofbiz on older database, doing a test upgrade to each and every new version, and then seeing what is different.  This is a *very* hard problem, not easy to automate, and takes smart people lots of time.  This is not something you can just force on the community.  No one has sat down to do this very busy, hard work, so it hasn't yet been done.
>
> If you run trunk, then it is up to you to solve any per-version upgrade problems.
>
> However, official releases should really have appropriate, detailed, upgrade instructions.

This sounds like a decent business plan...

If I remember right, upgrades are one way Compiere (or more specifically, Jorg Janke) made money early on (before the sell-out).

-David

Reply | Threaded
Open this post in threaded view
|

Re: Design improvements?

Adam Heath-2
In reply to this post by Adam Heath-2
On 09/22/2010 05:59 PM, Adam Heath wrote:

> On 09/22/2010 05:53 PM, Adam Heath wrote:
>> On 09/22/2010 05:44 PM, Jacques Le Roux wrote:
>>> From: "Adam Heath" <[hidden email]>
>>>> On 09/22/2010 03:33 PM, BJ Freeman wrote:
>>>>> Am it to gather by this that you can do a direct SVN update and all
>>>>> your
>>>>> customization continue to work?
>>>>> say from 9.04 to 10.04
>>>>
>>>> Ofbiz has never supported upgrades. I agree with BJ here.
>>>>
>>>> Database tables can change. Not all changes are automatic. Such
>>>> changes are not listed in a simple place(in the source).
>>>>
>>>> Values in tables can change. No upgrade conversions are
>>>> provided(again, in a simple place).
>>>
>>> Is this useless?
>>> https://cwiki.apache.org/confluence/display/OFBTECH/Revisions+Requiring+Data+Migration
>>>
>>>
>>
>> Pretty much. Way to hard to follow. Too much information. Needs more
>> automation.
>>
>> And I'm a fairly technical ofbiz guy. And where is this linked from?
>
> I should rephrase that.
>
> That document is too techincal, even for those who are quite familiar
> with the internals of ofbiz.
>
> People just doing plain jane installs will have no idea how to deal with
> that document.
>
> What we need to do, is become stupid. Forget everything you know. Then
> try to read that.
>
> The other issue is when did ofbiz last *officially* release? Waiting too
> long to release gives *large* changes that have to be done manually.
>
> Plus, if someone(brainfood or bj) has based an install off of an svn
> *snapshot*, then we can't expect the project to help us much when doing
> upgrade conversions.

Here are my thoughts on a more automagical way to do upgrades.

When a change is committed, the author may not be aware of the
implications of that change, as far as upgrades are concerned.  Even
if the author is aware, and they provide some kind of upgrade script,
the script itself could be buggy, so a new version would need to be
created.  However, you can't go back in time to fix the previous
script.  This means the upgrade scripts need to be maintained
out-of-band, while somehow referencing the previous change(by id, by
date, by version, or something).

Next, is that you don't need to be concerned with all possible
variations of upgrades.  A->B->C->D is all you need to be concerned
about.  There is no reason to have a script to go from A->D.

If ofbiz stores the installed version id somewhere(along with the date
*of the version*), and then compares the new version id(along with the
date), it can fetch the series of scripts nescessary to upgrade
incrementally.

My proposal, is for every commit that requires some upgrade support,
to have an external repository(could still be hosted on
svn.apache.org, could even be in each particular branch); let's say
$OFBIZ_HOME/upgrade/scripts/.  At that location, there will be a text
file registry, that lists the original revision id(doesn't have to be
the svn id, it's just completely freefrom), and the date of the
original id.  It would also list the script to run to do any upgrade
processing.

Ofbiz would then just run the scripts in series.  Or, the admin would.

This in-place upgrade system would probably *not* use much ofbiz code.
  Or, it would have to use a very small subset.  I'm not entirely
certain about how the system would actually function, other than the
high-level series of steps I've outlined.

Reply | Threaded
Open this post in threaded view
|

Re: Design improvements?

BJ Freeman
I think for the first cut I have simplified it both for the committer
and for the clients current version of ofbiz.
A webtools that goes to the Commit email archive and reads the commits
and compares then to the svn image of ofbiz and the Customize components
that are either in the component-ofbiz.xml or in hot-depoly.
the compare may be a little tricky but I think it is doable with regx
it will generate a report of its findings only.
at least that satisfies my current requirements to flag what I need to
look at.

the next cut would become more intelligent to suggest upgrades paths
based on the commit being processed.
or maybe using the commit number will look in the components patch
folder for a script for the upgrade provided by the committer.

or as you say have a location it goes to for updates that is not part of
the regular SVN image like the site is.
Could even put it under the site.

Adam Heath sent the following on 9/23/2010 9:33 AM:

> On 09/22/2010 05:59 PM, Adam Heath wrote:
>> On 09/22/2010 05:53 PM, Adam Heath wrote:
>>> On 09/22/2010 05:44 PM, Jacques Le Roux wrote:
>>>> From: "Adam Heath" <[hidden email]>
>>>>> On 09/22/2010 03:33 PM, BJ Freeman wrote:
>>>>>> Am it to gather by this that you can do a direct SVN update and all
>>>>>> your
>>>>>> customization continue to work?
>>>>>> say from 9.04 to 10.04
>>>>>
>>>>> Ofbiz has never supported upgrades. I agree with BJ here.
>>>>>
>>>>> Database tables can change. Not all changes are automatic. Such
>>>>> changes are not listed in a simple place(in the source).
>>>>>
>>>>> Values in tables can change. No upgrade conversions are
>>>>> provided(again, in a simple place).
>>>>
>>>> Is this useless?
>>>> https://cwiki.apache.org/confluence/display/OFBTECH/Revisions+Requiring+Data+Migration
>>>>
>>>>
>>>>
>>>
>>> Pretty much. Way to hard to follow. Too much information. Needs more
>>> automation.
>>>
>>> And I'm a fairly technical ofbiz guy. And where is this linked from?
>>
>> I should rephrase that.
>>
>> That document is too techincal, even for those who are quite familiar
>> with the internals of ofbiz.
>>
>> People just doing plain jane installs will have no idea how to deal with
>> that document.
>>
>> What we need to do, is become stupid. Forget everything you know. Then
>> try to read that.
>>
>> The other issue is when did ofbiz last *officially* release? Waiting too
>> long to release gives *large* changes that have to be done manually.
>>
>> Plus, if someone(brainfood or bj) has based an install off of an svn
>> *snapshot*, then we can't expect the project to help us much when doing
>> upgrade conversions.
>
> Here are my thoughts on a more automagical way to do upgrades.
>
> When a change is committed, the author may not be aware of the
> implications of that change, as far as upgrades are concerned. Even if
> the author is aware, and they provide some kind of upgrade script, the
> script itself could be buggy, so a new version would need to be created.
> However, you can't go back in time to fix the previous script. This
> means the upgrade scripts need to be maintained out-of-band, while
> somehow referencing the previous change(by id, by date, by version, or
> something).
>
> Next, is that you don't need to be concerned with all possible
> variations of upgrades. A->B->C->D is all you need to be concerned
> about. There is no reason to have a script to go from A->D.
>
> If ofbiz stores the installed version id somewhere(along with the date
> *of the version*), and then compares the new version id(along with the
> date), it can fetch the series of scripts nescessary to upgrade
> incrementally.
>
> My proposal, is for every commit that requires some upgrade support, to
> have an external repository(could still be hosted on svn.apache.org,
> could even be in each particular branch); let's say
> $OFBIZ_HOME/upgrade/scripts/. At that location, there will be a text
> file registry, that lists the original revision id(doesn't have to be
> the svn id, it's just completely freefrom), and the date of the original
> id. It would also list the script to run to do any upgrade processing.
>
> Ofbiz would then just run the scripts in series. Or, the admin would.
>
> This in-place upgrade system would probably *not* use much ofbiz code.
> Or, it would have to use a very small subset. I'm not entirely certain
> about how the system would actually function, other than the high-level
> series of steps I've outlined.
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Design improvements?

Jacques Le Roux
Administrator
You are ideas sounds both interesting, but I wonder if we will ever have enough commitment to do them...

Jacques

From: "BJ Freeman" <[hidden email]>

>I think for the first cut I have simplified it both for the committer
> and for the clients current version of ofbiz.
> A webtools that goes to the Commit email archive and reads the commits
> and compares then to the svn image of ofbiz and the Customize components
> that are either in the component-ofbiz.xml or in hot-depoly.
> the compare may be a little tricky but I think it is doable with regx
> it will generate a report of its findings only.
> at least that satisfies my current requirements to flag what I need to
> look at.
>
> the next cut would become more intelligent to suggest upgrades paths
> based on the commit being processed.
> or maybe using the commit number will look in the components patch
> folder for a script for the upgrade provided by the committer.
>
> or as you say have a location it goes to for updates that is not part of
> the regular SVN image like the site is.
> Could even put it under the site.
>
> Adam Heath sent the following on 9/23/2010 9:33 AM:
>> On 09/22/2010 05:59 PM, Adam Heath wrote:
>>> On 09/22/2010 05:53 PM, Adam Heath wrote:
>>>> On 09/22/2010 05:44 PM, Jacques Le Roux wrote:
>>>>> From: "Adam Heath" <[hidden email]>
>>>>>> On 09/22/2010 03:33 PM, BJ Freeman wrote:
>>>>>>> Am it to gather by this that you can do a direct SVN update and all
>>>>>>> your
>>>>>>> customization continue to work?
>>>>>>> say from 9.04 to 10.04
>>>>>>
>>>>>> Ofbiz has never supported upgrades. I agree with BJ here.
>>>>>>
>>>>>> Database tables can change. Not all changes are automatic. Such
>>>>>> changes are not listed in a simple place(in the source).
>>>>>>
>>>>>> Values in tables can change. No upgrade conversions are
>>>>>> provided(again, in a simple place).
>>>>>
>>>>> Is this useless?
>>>>> https://cwiki.apache.org/confluence/display/OFBTECH/Revisions+Requiring+Data+Migration
>>>>>
>>>>>
>>>>>
>>>>
>>>> Pretty much. Way to hard to follow. Too much information. Needs more
>>>> automation.
>>>>
>>>> And I'm a fairly technical ofbiz guy. And where is this linked from?
>>>
>>> I should rephrase that.
>>>
>>> That document is too techincal, even for those who are quite familiar
>>> with the internals of ofbiz.
>>>
>>> People just doing plain jane installs will have no idea how to deal with
>>> that document.
>>>
>>> What we need to do, is become stupid. Forget everything you know. Then
>>> try to read that.
>>>
>>> The other issue is when did ofbiz last *officially* release? Waiting too
>>> long to release gives *large* changes that have to be done manually.
>>>
>>> Plus, if someone(brainfood or bj) has based an install off of an svn
>>> *snapshot*, then we can't expect the project to help us much when doing
>>> upgrade conversions.
>>
>> Here are my thoughts on a more automagical way to do upgrades.
>>
>> When a change is committed, the author may not be aware of the
>> implications of that change, as far as upgrades are concerned. Even if
>> the author is aware, and they provide some kind of upgrade script, the
>> script itself could be buggy, so a new version would need to be created.
>> However, you can't go back in time to fix the previous script. This
>> means the upgrade scripts need to be maintained out-of-band, while
>> somehow referencing the previous change(by id, by date, by version, or
>> something).
>>
>> Next, is that you don't need to be concerned with all possible
>> variations of upgrades. A->B->C->D is all you need to be concerned
>> about. There is no reason to have a script to go from A->D.
>>
>> If ofbiz stores the installed version id somewhere(along with the date
>> *of the version*), and then compares the new version id(along with the
>> date), it can fetch the series of scripts nescessary to upgrade
>> incrementally.
>>
>> My proposal, is for every commit that requires some upgrade support, to
>> have an external repository(could still be hosted on svn.apache.org,
>> could even be in each particular branch); let's say
>> $OFBIZ_HOME/upgrade/scripts/. At that location, there will be a text
>> file registry, that lists the original revision id(doesn't have to be
>> the svn id, it's just completely freefrom), and the date of the original
>> id. It would also list the script to run to do any upgrade processing.
>>
>> Ofbiz would then just run the scripts in series. Or, the admin would.
>>
>> This in-place upgrade system would probably *not* use much ofbiz code.
>> Or, it would have to use a very small subset. I'm not entirely certain
>> about how the system would actually function, other than the high-level
>> series of steps I've outlined.
>>
>>
>

Reply | Threaded
Open this post in threaded view
|

Re: Design improvements?

BJ Freeman
Fools walk in where other fear to tread. LOL
It and setup are my two priorities at the moment.
like you I only have so much time, but I am plugging away at it.

I am sure if funding came about there were be many that would change
their priorities.

Jacques Le Roux sent the following on 9/26/2010 4:02 PM:

> You are ideas sounds both interesting, but I wonder if we will ever have
> enough commitment to do them...
>
> Jacques
>

> From: "BJ Freeman" <[hidden email]>
>> I think for the first cut I have simplified it both for the committer
>> and for the clients current version of ofbiz.
>> A webtools that goes to the Commit email archive and reads the commits
>> and compares then to the svn image of ofbiz and the Customize
>> components that are either in the component-ofbiz.xml or in hot-depoly.
>> the compare may be a little tricky but I think it is doable with regx
>> it will generate a report of its findings only.
>> at least that satisfies my current requirements to flag what I need to
>> look at.
>>
>> the next cut would become more intelligent to suggest upgrades paths
>> based on the commit being processed.
>> or maybe using the commit number will look in the components patch
>> folder for a script for the upgrade provided by the committer.
>>
>> or as you say have a location it goes to for updates that is not part
>> of the regular SVN image like the site is.
>> Could even put it under the site.
>>
>> Adam Heath sent the following on 9/23/2010 9:33 AM:
>>> On 09/22/2010 05:59 PM, Adam Heath wrote:
>>>> On 09/22/2010 05:53 PM, Adam Heath wrote:
>>>>> On 09/22/2010 05:44 PM, Jacques Le Roux wrote:
>>>>>> From: "Adam Heath" <[hidden email]>
>>>>>>> On 09/22/2010 03:33 PM, BJ Freeman wrote:
>>>>>>>> Am it to gather by this that you can do a direct SVN update and all
>>>>>>>> your
>>>>>>>> customization continue to work?
>>>>>>>> say from 9.04 to 10.04
>>>>>>>
>>>>>>> Ofbiz has never supported upgrades. I agree with BJ here.
>>>>>>>
>>>>>>> Database tables can change. Not all changes are automatic. Such
>>>>>>> changes are not listed in a simple place(in the source).
>>>>>>>
>>>>>>> Values in tables can change. No upgrade conversions are
>>>>>>> provided(again, in a simple place).
>>>>>>
>>>>>> Is this useless?
>>>>>> https://cwiki.apache.org/confluence/display/OFBTECH/Revisions+Requiring+Data+Migration
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> Pretty much. Way to hard to follow. Too much information. Needs more
>>>>> automation.
>>>>>
>>>>> And I'm a fairly technical ofbiz guy. And where is this linked from?
>>>>
>>>> I should rephrase that.
>>>>
>>>> That document is too techincal, even for those who are quite familiar
>>>> with the internals of ofbiz.
>>>>
>>>> People just doing plain jane installs will have no idea how to deal
>>>> with
>>>> that document.
>>>>
>>>> What we need to do, is become stupid. Forget everything you know. Then
>>>> try to read that.
>>>>
>>>> The other issue is when did ofbiz last *officially* release? Waiting
>>>> too
>>>> long to release gives *large* changes that have to be done manually.
>>>>
>>>> Plus, if someone(brainfood or bj) has based an install off of an svn
>>>> *snapshot*, then we can't expect the project to help us much when doing
>>>> upgrade conversions.
>>>
>>> Here are my thoughts on a more automagical way to do upgrades.
>>>
>>> When a change is committed, the author may not be aware of the
>>> implications of that change, as far as upgrades are concerned. Even if
>>> the author is aware, and they provide some kind of upgrade script, the
>>> script itself could be buggy, so a new version would need to be created.
>>> However, you can't go back in time to fix the previous script. This
>>> means the upgrade scripts need to be maintained out-of-band, while
>>> somehow referencing the previous change(by id, by date, by version, or
>>> something).
>>>
>>> Next, is that you don't need to be concerned with all possible
>>> variations of upgrades. A->B->C->D is all you need to be concerned
>>> about. There is no reason to have a script to go from A->D.
>>>
>>> If ofbiz stores the installed version id somewhere(along with the date
>>> *of the version*), and then compares the new version id(along with the
>>> date), it can fetch the series of scripts nescessary to upgrade
>>> incrementally.
>>>
>>> My proposal, is for every commit that requires some upgrade support, to
>>> have an external repository(could still be hosted on svn.apache.org,
>>> could even be in each particular branch); let's say
>>> $OFBIZ_HOME/upgrade/scripts/. At that location, there will be a text
>>> file registry, that lists the original revision id(doesn't have to be
>>> the svn id, it's just completely freefrom), and the date of the original
>>> id. It would also list the script to run to do any upgrade processing.
>>>
>>> Ofbiz would then just run the scripts in series. Or, the admin would.
>>>
>>> This in-place upgrade system would probably *not* use much ofbiz code.
>>> Or, it would have to use a very small subset. I'm not entirely certain
>>> about how the system would actually function, other than the high-level
>>> series of steps I've outlined.
>>>
>>>
>>
>
>
12