How manage datamodel change
Locked
 
|
Many
times
when I try
to refactor some code, I discovered
some entities
that need to be refactored
in
order to
homogenise or improve them
with the new framework
functionnalities.
We
have a page to describe how to do this
migration for each modification https://cwiki.apache.org/confluence/display/OFBIZ/Revisions+Requiring+Data+Migration+-+upgrade+ofbiz
I
think that this is
not easy for an end
user nor
useful with the OFBiz
release workflow.
Maybe
we can improve it with
this :
*
Organize
this page by OFBiz
release branch since
we don't change the data model on stable (all the
release)
branch.
When
we announce
the availability of
a new release branch, we can fix the data model changes
between this branch
and the previous one
*
Create a state table to indicate some information between the OFBiz
and the database, I think about the OFBiz
release that exploit this database
*
Store each datamodel migration
script into a
specific directory for each component. Those
scripts
describe the migration through
SQL
scripts,
dedicated
services
or any
automated
operations
needed to
achieve the migration. (and maybe we should use plugins
for that)
*
Create a new build target to apply the upgraded
datamodel
**
check the database OFBiz release
**
apply migration scripts
for each component
**
increase the database OFBiz
release
If
we have a more
efficient datamodel
modification
workflow, we can
improve and homogenise our historical datamodel with serenity.
WDYT ?
--
|
|
Hi Nicolas,
Very nice ideas. You made me think of a simple and beautiful solution based on your mentioning of plugins: For example, you create a plugin called UpgradeToOfbiz16 and UpgradeToOfbiz17 etc ... and this plugin when installed using ./gradlew installPlugin -PpluginName=UpgradeToOfbiz16 would take care of everything :) so this would be things like: - changes to the database. maybe we can even use something like liquibase - copying / adding / deleting - installing or uninstalling other plugins So someone takes care of all the complexity and users just install another plugin :) I don't know if this is sort of what you have in mind? Taher Alkhateeb On Friday, 29 July 2016, Nicolas Malin <[hidden email]> wrote: > Many times when I try to refactor some code, I discovered some entities > that need to be refactored in order to homogenise or improve them with > the new framework functionnalities. > > We have a page to describe how to do this migration for each modification > https://cwiki.apache.org/confluence/display/OFBIZ/Revisions+Requiring+Data+Migration+-+upgrade+ofbiz > > I think that this is not easy for an end user nor useful with the OFBiz > release workflow. > Maybe we can improve it with this : > * Organize this page by OFBiz release branch since we don't change the > data model on stable (all the release) branch. When we announce the > availability of a new release branch, we can fix the data model changes > between this branch and the previous one > * Create a state table to indicate some information between the OFBiz and > the database, I think about the OFBiz release that exploit this database > * Store each datamodel migration script into a specific directory for > each component. Those scripts describe the migration through SQL scripts, > dedicated services or any automated operations needed to achieve the > migration. (and maybe we should use plugins for that) > * Create a new build target to apply the upgraded datamodel > ** check the database OFBiz release > ** apply migration scripts for each component > ** increase the database OFBiz release > > If we have a more efficient datamodel modification workflow, we can > improve and homogenise our historical datamodel with serenity. > > WDYT ? > > > -- > [image: logoNrd] <http://nereide.fr/> > Nicolas Malin > The apache way <http://theapacheway.com/> : *Openness* Technical > decisions are made publicly > [hidden email] > <javascript:_e(%7B%7D,'cvml','[hidden email]');> > 8 rue des Déportés 37000 TOURS, 02 47 50 30 54 > Apache OFBiz <http://ofbiz.apache.org/> | The Apache Way > <http://theapacheway.com/> | ofbiz-fr <http://www.ofbiz-fr.org/> | réseau > LE <http://www.libre-entreprise.org/> > |
Re: How manage datamodel change
|
|
Yes Taher your solution would be nice, just when i see it, I have the
addon syndrom in my mind :) From my vision, I see a solution who each component can expose what he change : * application/datamodel/upgrade/release17/update.groovy * framework/service/upgrade/release17/update.groovy * hot-deploy/myowncomponent/upgrade/release17/update.groovy When I install a new ofbiz from release16 to release17 I can run : * ./gradlew upgrade -> OFBiz start -> OFBiz check the database version if it's under release17 -> For each component call upgrade/release17/update.groovy on the same transaction But sure if we cross own ideas we can find THE solution :) Nicolas Le 29/07/2016 à 18:36, Taher Alkhateeb a écrit : > Hi Nicolas, > > Very nice ideas. You made me think of a simple and beautiful solution based > on your mentioning of plugins: > > For example, you create a plugin called UpgradeToOfbiz16 and > UpgradeToOfbiz17 etc ... and this plugin when installed using ./gradlew > installPlugin -PpluginName=UpgradeToOfbiz16 would take care of everything > :) so this would be things like: > - changes to the database. maybe we can even use something like liquibase > - copying / adding / deleting > - installing or uninstalling other plugins > > So someone takes care of all the complexity and users just install another > plugin :) I don't know if this is sort of what you have in mind? > > Taher Alkhateeb > > On Friday, 29 July 2016, Nicolas Malin <[hidden email]> wrote: > >> Many times when I try to refactor some code, I discovered some entities >> that need to be refactored in order to homogenise or improve them with >> the new framework functionnalities. >> >> We have a page to describe how to do this migration for each modification >> https://cwiki.apache.org/confluence/display/OFBIZ/Revisions+Requiring+Data+Migration+-+upgrade+ofbiz >> >> I think that this is not easy for an end user nor useful with the OFBiz >> release workflow. >> Maybe we can improve it with this : >> * Organize this page by OFBiz release branch since we don't change the >> data model on stable (all the release) branch. When we announce the >> availability of a new release branch, we can fix the data model changes >> between this branch and the previous one >> * Create a state table to indicate some information between the OFBiz and >> the database, I think about the OFBiz release that exploit this database >> * Store each datamodel migration script into a specific directory for >> each component. Those scripts describe the migration through SQL scripts, >> dedicated services or any automated operations needed to achieve the >> migration. (and maybe we should use plugins for that) >> * Create a new build target to apply the upgraded datamodel >> ** check the database OFBiz release >> ** apply migration scripts for each component >> ** increase the database OFBiz release >> >> If we have a more efficient datamodel modification workflow, we can >> improve and homogenise our historical datamodel with serenity. >> >> WDYT ? >> >> >> -- >> [image: logoNrd] <http://nereide.fr/> >> Nicolas Malin >> The apache way <http://theapacheway.com/> : *Openness* Technical >> decisions are made publicly >> [hidden email] >> <javascript:_e(%7B%7D,'cvml','[hidden email]');> >> 8 rue des Déportés 37000 TOURS, 02 47 50 30 54 >> Apache OFBiz <http://ofbiz.apache.org/> | The Apache Way >> <http://theapacheway.com/> | ofbiz-fr <http://www.ofbiz-fr.org/> | réseau >> LE <http://www.libre-entreprise.org/> >> |
|
|
Hmm I see,
So would these files ship with each release? Or you just keep adding more directories for each release in each component in trunk? Taher Alkhateeb On Jul 29, 2016 8:48 PM, "Nicolas Malin" <[hidden email]> wrote: > Yes Taher your solution would be nice, just when i see it, I have the > addon syndrom in my mind :) > > From my vision, I see a solution who each component can expose what he > change : > > * application/datamodel/upgrade/release17/update.groovy > * framework/service/upgrade/release17/update.groovy > * hot-deploy/myowncomponent/upgrade/release17/update.groovy > > When I install a new ofbiz from release16 to release17 I can run : > * ./gradlew upgrade > -> OFBiz start > -> OFBiz check the database version if it's under release17 > -> For each component call upgrade/release17/update.groovy on the same > transaction > > But sure if we cross own ideas we can find THE solution :) > > Nicolas > > Le 29/07/2016 à 18:36, Taher Alkhateeb a écrit : > >> Hi Nicolas, >> >> Very nice ideas. You made me think of a simple and beautiful solution >> based >> on your mentioning of plugins: >> >> For example, you create a plugin called UpgradeToOfbiz16 and >> UpgradeToOfbiz17 etc ... and this plugin when installed using ./gradlew >> installPlugin -PpluginName=UpgradeToOfbiz16 would take care of everything >> :) so this would be things like: >> - changes to the database. maybe we can even use something like liquibase >> - copying / adding / deleting >> - installing or uninstalling other plugins >> >> So someone takes care of all the complexity and users just install another >> plugin :) I don't know if this is sort of what you have in mind? >> >> Taher Alkhateeb >> >> On Friday, 29 July 2016, Nicolas Malin <[hidden email]> wrote: >> >> Many times when I try to refactor some code, I discovered some entities >>> that need to be refactored in order to homogenise or improve them with >>> the new framework functionnalities. >>> >>> We have a page to describe how to do this migration for each modification >>> >>> https://cwiki.apache.org/confluence/display/OFBIZ/Revisions+Requiring+Data+Migration+-+upgrade+ofbiz >>> >>> I think that this is not easy for an end user nor useful with the OFBiz >>> release workflow. >>> Maybe we can improve it with this : >>> * Organize this page by OFBiz release branch since we don't change the >>> data model on stable (all the release) branch. When we announce the >>> availability of a new release branch, we can fix the data model changes >>> between this branch and the previous one >>> * Create a state table to indicate some information between the OFBiz and >>> the database, I think about the OFBiz release that exploit this database >>> * Store each datamodel migration script into a specific directory for >>> each component. Those scripts describe the migration through SQL scripts, >>> dedicated services or any automated operations needed to achieve the >>> migration. (and maybe we should use plugins for that) >>> * Create a new build target to apply the upgraded datamodel >>> ** check the database OFBiz release >>> ** apply migration scripts for each component >>> ** increase the database OFBiz release >>> >>> If we have a more efficient datamodel modification workflow, we can >>> improve and homogenise our historical datamodel with serenity. >>> >>> WDYT ? >>> >>> >>> -- >>> [image: logoNrd] <http://nereide.fr/> >>> Nicolas Malin >>> The apache way <http://theapacheway.com/> : *Openness* Technical >>> decisions are made publicly >>> [hidden email] >>> <javascript:_e(%7B%7D,'cvml','[hidden email]');> >>> 8 rue des Déportés 37000 TOURS, 02 47 50 30 54 >>> Apache OFBiz <http://ofbiz.apache.org/> | The Apache Way >>> <http://theapacheway.com/> | ofbiz-fr <http://www.ofbiz-fr.org/> | >>> réseau >>> LE <http://www.libre-entreprise.org/> >>> >>> > |
Re: How manage datamodel change
Administrator
|
In reply to this post by Nicolas Malin-2
Hi Nicolas,
Inline Le 29/07/2016 à 18:08, Nicolas Malin a écrit : > Many timeswhen I tryto refactor some code, I discoveredsome entitiesthat need to be refactoredin order to homogenise or improve themwith the new > framework functionnalities. > > We have a page to describe how to do thismigration for each modification > https://cwiki.apache.org/confluence/display/OFBIZ/Revisions+Requiring+Data+Migration+-+upgrade+ofbiz > > I think that this is not easy for an end user noruseful with the OFBiz release workflow. > Maybe we can improve it withthis : > * Organizethis page by OFBiz releasebranchsincewe don't change the data model on stable(all the release)branch.When we announce the availability ofa > new release branch, we can fix the data model changesbetween this branch and the previous one +1 > * Create a state table to indicate some information between the OFBizand the database, I think about the OFBiz release that exploit this database +0, sorry not sure what it means, else do we really need that? What for? > * Store each datamodel migrationscript intoaspecific directory for each component. Those scriptsdescribe the migration throughSQLscripts, > dedicatedservicesor anyautomatedoperationsneeded to achieve the migration.(and maybe we should use plugins for that) Sounds like a good idea +1 > * Create a new build target to apply the upgradeddatamodel > ** check the database OFBiz release > ** apply migration scriptsfor each component > ** increase the database OFBiz release Would be wonderful +1 Jacques > > If we have a more efficient datamodel modificationworkflow, we can improve and homogenise our historical datamodel with serenity. > > WDYT? > > > -- > logoNrd <http://nereide.fr/> > Nicolas Malin > The apache way <http://theapacheway.com/> : *Openness* Technical decisions are made publicly > [hidden email] > 8 rue des Déportés 37000 TOURS, 02 47 50 30 54 > > Apache OFBiz <http://ofbiz.apache.org/> | The Apache Way <http://theapacheway.com/> | ofbiz-fr <http://www.ofbiz-fr.org/> | réseau LE > <http://www.libre-entreprise.org/> |
Re: How manage datamodel change
|
Administrator
|
In reply to this post by taher
Le 29/07/2016 à 20:23, Taher Alkhateeb a écrit :
> Hmm I see, > > So would these files ship with each release? Or you just keep adding more > directories for each release in each component in trunk? Good question :) Jacques > > Taher Alkhateeb > > On Jul 29, 2016 8:48 PM, "Nicolas Malin" <[hidden email]> wrote: > >> Yes Taher your solution would be nice, just when i see it, I have the >> addon syndrom in my mind :) >> >> From my vision, I see a solution who each component can expose what he >> change : >> >> * application/datamodel/upgrade/release17/update.groovy >> * framework/service/upgrade/release17/update.groovy >> * hot-deploy/myowncomponent/upgrade/release17/update.groovy >> >> When I install a new ofbiz from release16 to release17 I can run : >> * ./gradlew upgrade >> -> OFBiz start >> -> OFBiz check the database version if it's under release17 >> -> For each component call upgrade/release17/update.groovy on the same >> transaction >> >> But sure if we cross own ideas we can find THE solution :) >> >> Nicolas >> >> Le 29/07/2016 à 18:36, Taher Alkhateeb a écrit : >> >>> Hi Nicolas, >>> >>> Very nice ideas. You made me think of a simple and beautiful solution >>> based >>> on your mentioning of plugins: >>> >>> For example, you create a plugin called UpgradeToOfbiz16 and >>> UpgradeToOfbiz17 etc ... and this plugin when installed using ./gradlew >>> installPlugin -PpluginName=UpgradeToOfbiz16 would take care of everything >>> :) so this would be things like: >>> - changes to the database. maybe we can even use something like liquibase >>> - copying / adding / deleting >>> - installing or uninstalling other plugins >>> >>> So someone takes care of all the complexity and users just install another >>> plugin :) I don't know if this is sort of what you have in mind? >>> >>> Taher Alkhateeb >>> >>> On Friday, 29 July 2016, Nicolas Malin <[hidden email]> wrote: >>> >>> Many times when I try to refactor some code, I discovered some entities >>>> that need to be refactored in order to homogenise or improve them with >>>> the new framework functionnalities. >>>> >>>> We have a page to describe how to do this migration for each modification >>>> >>>> https://cwiki.apache.org/confluence/display/OFBIZ/Revisions+Requiring+Data+Migration+-+upgrade+ofbiz >>>> >>>> I think that this is not easy for an end user nor useful with the OFBiz >>>> release workflow. >>>> Maybe we can improve it with this : >>>> * Organize this page by OFBiz release branch since we don't change the >>>> data model on stable (all the release) branch. When we announce the >>>> availability of a new release branch, we can fix the data model changes >>>> between this branch and the previous one >>>> * Create a state table to indicate some information between the OFBiz and >>>> the database, I think about the OFBiz release that exploit this database >>>> * Store each datamodel migration script into a specific directory for >>>> each component. Those scripts describe the migration through SQL scripts, >>>> dedicated services or any automated operations needed to achieve the >>>> migration. (and maybe we should use plugins for that) >>>> * Create a new build target to apply the upgraded datamodel >>>> ** check the database OFBiz release >>>> ** apply migration scripts for each component >>>> ** increase the database OFBiz release >>>> >>>> If we have a more efficient datamodel modification workflow, we can >>>> improve and homogenise our historical datamodel with serenity. >>>> >>>> WDYT ? >>>> >>>> >>>> -- >>>> [image: logoNrd] <http://nereide.fr/> >>>> Nicolas Malin >>>> The apache way <http://theapacheway.com/> : *Openness* Technical >>>> decisions are made publicly >>>> [hidden email] >>>> <javascript:_e(%7B%7D,'cvml','[hidden email]');> >>>> 8 rue des Déportés 37000 TOURS, 02 47 50 30 54 >>>> Apache OFBiz <http://ofbiz.apache.org/> | The Apache Way >>>> <http://theapacheway.com/> | ofbiz-fr <http://www.ofbiz-fr.org/> | >>>> réseau >>>> LE <http://www.libre-entreprise.org/> >>>> >>>> |
Re: How manage datamodel change
|
|
Hi, I'd rather imagine something like : $ gradlew searchPlugin migrate* migrate12.04-13.07 migrate13.07-14.12 ... $ gradlew installPlugin migrate13.07-14.12 So not delivered with the release, but available from plugin system. My 2 cts Gil Le 30/07/2016 à 11:17, Jacques Le Roux
a écrit :
Le 29/07/2016 à 20:23, Taher Alkhateeb a écrit : |
|
|
Hi Gil, All
Yeah I like that idea. Even the words you used are clearer. It is really a migration more than an update. So the migration happens after the update. This leaves us with custom components that Nicolas mentioned which we can also consider as plugins (if we decide te replace hot-deploy with plugins). So perhaps one solution is to call: ./gradlew migratePlugin -PpluginName=mycustomplugin -PpluginVersion=desiredversion However plugin migrations are independent of OFBiz migrations so we can just consider it part of the plugin update process. So maybe the below command both updates the plugin to the new version and applies migration: ./gradlew updatePlugin -PpluginName=mycustomplugin -PpluginVersion=desiredversion -PmigratePlugin=true All ideas at the moment, but I think they sound cool. Taher Alkhateeb On Saturday, 30 July 2016, gil portenseigne <[hidden email]> wrote: > > Hi, > > I'd rather imagine something like : > > $ gradlew searchPlugin migrate* > > migrate12.04-13.07 > > migrate13.07-14.12 > > ... > > $ gradlew installPlugin migrate13.07-14.12 > > So not delivered with the release, but available from plugin system. > > My 2 cts > > Gil > > > Le 30/07/2016 à 11:17, Jacques Le Roux a écrit : >> >> Le 29/07/2016 à 20:23, Taher Alkhateeb a écrit : >>> >>> Hmm I see, >>> >>> So would these files ship with each release? Or you just keep adding more >>> directories for each release in each component in trunk? >> >> >> Good question :) >> >> Jacques >> >>> >>> Taher Alkhateeb >>> >>> On Jul 29, 2016 8:48 PM, "Nicolas Malin" <[hidden email]> wrote: >>> >>>> Yes Taher your solution would be nice, just when i see it, I have the >>>> addon syndrom in my mind :) >>>> >>>> From my vision, I see a solution who each component can expose what he >>>> change : >>>> >>>> * application/datamodel/upgrade/release17/update.groovy >>>> * framework/service/upgrade/release17/update.groovy >>>> * hot-deploy/myowncomponent/upgrade/release17/update.groovy >>>> >>>> When I install a new ofbiz from release16 to release17 I can run : >>>> * ./gradlew upgrade >>>> -> OFBiz start >>>> -> OFBiz check the database version if it's under release17 >>>> -> For each component call upgrade/release17/update.groovy on the same >>>> transaction >>>> >>>> But sure if we cross own ideas we can find THE solution :) >>>> >>>> Nicolas >>>> >>>> Le 29/07/2016 à 18:36, Taher Alkhateeb a écrit : >>>> >>>>> Hi Nicolas, >>>>> >>>>> Very nice ideas. You made me think of a simple and beautiful solution >>>>> based >>>>> on your mentioning of plugins: >>>>> >>>>> For example, you create a plugin called UpgradeToOfbiz16 and >>>>> UpgradeToOfbiz17 etc ... and this plugin when installed using ./gradlew >>>>> installPlugin -PpluginName=UpgradeToOfbiz16 would take care of everything >>>>> :) so this would be things like: >>>>> - changes to the database. maybe we can even use something like liquibase >>>>> - copying / adding / deleting >>>>> - installing or uninstalling other plugins >>>>> >>>>> So someone takes care of all the complexity and users just install another >>>>> plugin :) I don't know if this is sort of what you have in mind? >>>>> >>>>> Taher Alkhateeb >>>>> >>>>> On Friday, 29 July 2016, Nicolas Malin <[hidden email]> wrote: >>>>> >>>>> Many times when I try to refactor some code, I discovered some entities >>>>>> >>>>>> that need to be refactored in order to homogenise or improve them with >>>>>> the new framework functionnalities. >>>>>> >>>>>> We have a page to describe how to do this migration for each modification >>>>>> >>>>>> https://cwiki.apache.org/confluence/display/OFBIZ/Revisions+Requiring+Data+Migration+-+upgrade+ofbiz <https://cwiki.apache.org/confluence/display/OFBIZ/Revisions+Requiring+Data+Migration+-+upgrade+ofbiz> >>>>>> >>>>>> I think that this is not easy for an end user nor useful with the OFBiz >>>>>> release workflow. >>>>>> Maybe we can improve it with this : >>>>>> * Organize this page by OFBiz release branch since we don't change the >>>>>> data model on stable (all the release) branch. When we announce the >>>>>> availability of a new release branch, we can fix the data model changes >>>>>> between this branch and the previous one >>>>>> * Create a state table to indicate some information between the OFBiz and >>>>>> the database, I think about the OFBiz release that exploit this database >>>>>> * Store each datamodel migration script into a specific directory for >>>>>> each component. Those scripts describe the migration through SQL scripts, >>>>>> dedicated services or any automated operations needed to achieve the >>>>>> migration. (and maybe we should use plugins for that) >>>>>> * Create a new build target to apply the upgraded datamodel >>>>>> ** check the database OFBiz release >>>>>> ** apply migration scripts for each component >>>>>> ** increase the database OFBiz release >>>>>> >>>>>> If we have a more efficient datamodel modification workflow, we can >>>>>> improve and homogenise our historical datamodel with serenity. >>>>>> >>>>>> WDYT ? >>>>>> >>>>>> >>>>>> -- >>>>>> [image: logoNrd] <http://nereide.fr/> <http://nereide.fr/> >>>>>> Nicolas Malin >>>>>> The apache way <http://theapacheway.com/> <http://theapacheway.com/> >>>>>> decisions are made publicly >>>>>> [hidden email] >>>>>> <javascript:_e(%7B%7D,'cvml','[hidden email]');> >>>>>> 8 rue des Déportés 37000 TOURS, 02 47 50 30 54 >>>>>> Apache OFBiz <http://ofbiz.apache.org/> <http://ofbiz.apache.org/> | The Apache Way >>>>>> <http://theapacheway.com/> <http://theapacheway.com/> | ofbiz-fr <http://www.ofbiz-fr.org/> <http://www.ofbiz-fr.org/> | >>>>>> réseau >>>>>> LE <http://www.libre-entreprise.org/> <http://www.libre-entreprise.org/> >>>>>> >>>>>> >> > |
«
Return to OFBiz - Dev
|
1 view|%1 views
| Free forum by Nabble | Edit this page |