OFBiz › OFBiz - Dev

Discussion: Seed and Demo Data Files

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

14 messages Options

Adrian Crum

Discussion: Seed and Demo Data Files

I'm getting ready to deploy the latest trunk to our production server,
and I have encountered a few problems with the way the seed data is set up.

In order to get the latest features to work with my existing database, I
have to run ant run-install-seed. The problem is, there is a lot of demo
data in the seed files. And I really mean a LOT. After running
run-install-seed, my database contains a lot of demo data that I had
previously removed.

I would like to suggest the following best practices:

1. Keep seed-initial, seed, and demo data in separate XML files.
2. Name the files accordingly, so their purpose is clear. Example:
[component name]InitialData.xml, [component name]SeedData.xml, and
[component name]DemoData.xml.

Right now I'm working on sorting out the data for the accounting
component. The AccountingSecurityData.xml file is loaded as seed data,
but it contains both seed AND demo data: security permissions are seed
data, assigning those permissions to demo security groups is demo data.
The AccountingTypeData.xml file is loaded as seed data, but it also
contains both seed and demo data. Those are just two examples.

I can commit some of my work as an example of what changes need to be made.

What do you think?

-Adrian

Pranay Pandey-2

Re: Discussion: Seed and Demo Data Files

Hi Adrian,

The data which is client specific should go in ext, so When we say run-
install-extseed it will go with seed data which is required for ext
data.

Adding suffix like you suggested is a good idea IMO. Lets see what
others have to say.

Thanks & Regards
--
Pranay Pandey
HotWax Media | http://www.hotwaxmedia.com

On Jun 4, 2009, at 10:41 PM, Adrian Crum wrote:

> I'm getting ready to deploy the latest trunk to our production
> server, and I have encountered a few problems with the way the seed
> data is set up.
>
> In order to get the latest features to work with my existing
> database, I have to run ant run-install-seed. The problem is, there
> is a lot of demo data in the seed files. And I really mean a LOT.
> After running run-install-seed, my database contains a lot of demo
> data that I had previously removed.
>
> I would like to suggest the following best practices:
>
> 1. Keep seed-initial, seed, and demo data in separate XML files.
> 2. Name the files accordingly, so their purpose is clear. Example:
> [component name]InitialData.xml, [component name]SeedData.xml, and
> [component name]DemoData.xml.
>
> Right now I'm working on sorting out the data for the accounting
> component. The AccountingSecurityData.xml file is loaded as seed
> data, but it contains both seed AND demo data: security permissions
> are seed data, assigning those permissions to demo security groups
> is demo data. The AccountingTypeData.xml file is loaded as seed
> data, but it also contains both seed and demo data. Those are just
> two examples.
>
> I can commit some of my work as an example of what changes need to
> be made.
>
> What do you think?
>
> -Adrian

Adrian Crum-2

Re: Discussion: Seed and Demo Data Files

In reply to this post by Adrian Crum

I wasn't referring to client-specific data. I was referring to Out-Of-the-box OFBiz data only.

-Adrian

--- On Thu, 6/4/09, Pranay Pandey <[hidden email]> wrote:

> From: Pranay Pandey <[hidden email]>
> Subject: Re: Discussion: Seed and Demo Data Files
> To: [hidden email]
> Cc: "Pranay Pandey" <[hidden email]>
> Date: Thursday, June 4, 2009, 11:40 PM
> Hi Adrian,
>
> The data which is client specific should go in ext, so When
> we say run-install-extseed it will go with seed data which
> is required for ext data.
>
> Adding suffix like you suggested is a good idea IMO. Lets
> see what others have to say.
>
>
> Thanks & Regards
> --
> Pranay Pandey
> HotWax Media | http://www.hotwaxmedia.com
>
>
> On Jun 4, 2009, at 10:41 PM, Adrian Crum wrote:
>
> > I'm getting ready to deploy the latest trunk to our
> production server, and I have encountered a few problems
> with the way the seed data is set up.
> >
> > In order to get the latest features to work with my
> existing database, I have to run ant run-install-seed. The
> problem is, there is a lot of demo data in the seed files.
> And I really mean a LOT. After running run-install-seed, my
> database contains a lot of demo data that I had previously
> removed.
> >
> > I would like to suggest the following best practices:
> >
> > 1. Keep seed-initial, seed, and demo data in separate
> XML files.
> > 2. Name the files accordingly, so their purpose is
> clear. Example: [component name]InitialData.xml, [component
> name]SeedData.xml, and [component name]DemoData.xml.
> >
> > Right now I'm working on sorting out the data for the
> accounting component. The AccountingSecurityData.xml file is
> loaded as seed data, but it contains both seed AND demo
> data: security permissions are seed data, assigning those
> permissions to demo security groups is demo data. The
> AccountingTypeData.xml file is loaded as seed data, but it
> also contains both seed and demo data. Those are just two
> examples.
> >
> > I can commit some of my work as an example of what
> changes need to be made.
> >
> > What do you think?
> >
> > -Adrian
>
>

Jacques Le Roux

Re: Discussion: Seed and Demo Data Files

Administrator

In reply to this post by Adrian Crum

+1 for differentiating data files names
+1 for good examples commited (or if easier shown somewhere)

Jacques

From: "Adrian Crum" <[hidden email]>

> I'm getting ready to deploy the latest trunk to our production server,
> and I have encountered a few problems with the way the seed data is set up.
>
> In order to get the latest features to work with my existing database, I
> have to run ant run-install-seed. The problem is, there is a lot of demo
> data in the seed files. And I really mean a LOT. After running
> run-install-seed, my database contains a lot of demo data that I had
> previously removed.
>
> I would like to suggest the following best practices:
>
> 1. Keep seed-initial, seed, and demo data in separate XML files.
> 2. Name the files accordingly, so their purpose is clear. Example:
> [component name]InitialData.xml, [component name]SeedData.xml, and
> [component name]DemoData.xml.
>
> Right now I'm working on sorting out the data for the accounting
> component. The AccountingSecurityData.xml file is loaded as seed data,
> but it contains both seed AND demo data: security permissions are seed
> data, assigning those permissions to demo security groups is demo data.
> The AccountingTypeData.xml file is loaded as seed data, but it also
> contains both seed and demo data. Those are just two examples.
>
> I can commit some of my work as an example of what changes need to be made.
>
> What do you think?
>
> -Adrian
>

Adrian Crum-2

Re: Discussion: Seed and Demo Data Files

In reply to this post by Adrian Crum

I was about to commit the change in the Asset Maintenance component (that our company uses OOTB) - but I realized that the change would have to include framework component changes. I ended up dropping the effort.

The bottom line is - it's an issue that has to be addressed (similar to the BigDecimal or the HTTP security issues we recently addressed) - but I'm not going to push it unless others see the benefit.

-Adrian

--- On Fri, 6/5/09, Jacques Le Roux <[hidden email]> wrote:

> From: Jacques Le Roux <[hidden email]>
> Subject: Re: Discussion: Seed and Demo Data Files
> To: [hidden email]
> Date: Friday, June 5, 2009, 12:06 AM
> +1 for differentiating data files
> names
> +1 for good examples commited (or if easier shown
> somewhere)
>
> Jacques
>
> From: "Adrian Crum" <[hidden email]>
> > I'm getting ready to deploy the latest trunk to our
> production server, and I have encountered a few problems
> with the way the seed data is set up.
> >
> > In order to get the latest features to work with my
> existing database, I have to run ant run-install-seed. The
> problem is, there is a lot of demo data in the seed files.
> And I really mean a LOT. After running run-install-seed, my
> database contains a lot of demo data that I had previously
> removed.
> >
> > I would like to suggest the following best practices:
> >
> > 1. Keep seed-initial, seed, and demo data in separate
> XML files.
> > 2. Name the files accordingly, so their purpose is
> clear. Example: [component name]InitialData.xml, [component
> name]SeedData.xml, and [component name]DemoData.xml.
> >
> > Right now I'm working on sorting out the data for the
> accounting component. The AccountingSecurityData.xml file is
> loaded as seed data, but it contains both seed AND demo
> data: security permissions are seed data, assigning those
> permissions to demo security groups is demo data. The
> AccountingTypeData.xml file is loaded as seed data, but it
> also contains both seed and demo data. Those are just two
> examples.
> >
> > I can commit some of my work as an example of what
> changes need to be made.
> >
> > What do you think?
> >
> > -Adrian
> >
>
>

Jacques Le Roux

Re: Discussion: Seed and Demo Data Files

Administrator

Hopefully the persons who commited these demo data in seed will follow your advice and we will end up without such problems in
future...

Thanks

Jacques

From: "Adrian Crum" <[hidden email]>

> I was about to commit the change in the Asset Maintenance component (that our company uses OOTB) - but I realized that the change
> would have to include framework component changes. I ended up dropping the effort.
>
> The bottom line is - it's an issue that has to be addressed (similar to the BigDecimal or the HTTP security issues we recently
> addressed) - but I'm not going to push it unless others see the benefit.
>
> -Adrian
>
>
> --- On Fri, 6/5/09, Jacques Le Roux <[hidden email]> wrote:
>
>> From: Jacques Le Roux <[hidden email]>
>> Subject: Re: Discussion: Seed and Demo Data Files
>> To: [hidden email]
>> Date: Friday, June 5, 2009, 12:06 AM
>> +1 for differentiating data files
>> names
>> +1 for good examples commited (or if easier shown
>> somewhere)
>>
>> Jacques
>>
>> From: "Adrian Crum" <[hidden email]>
>> > I'm getting ready to deploy the latest trunk to our
>> production server, and I have encountered a few problems
>> with the way the seed data is set up.
>> >
>> > In order to get the latest features to work with my
>> existing database, I have to run ant run-install-seed. The
>> problem is, there is a lot of demo data in the seed files.
>> And I really mean a LOT. After running run-install-seed, my
>> database contains a lot of demo data that I had previously
>> removed.
>> >
>> > I would like to suggest the following best practices:
>> >
>> > 1. Keep seed-initial, seed, and demo data in separate
>> XML files.
>> > 2. Name the files accordingly, so their purpose is
>> clear. Example: [component name]InitialData.xml, [component
>> name]SeedData.xml, and [component name]DemoData.xml.
>> >
>> > Right now I'm working on sorting out the data for the
>> accounting component. The AccountingSecurityData.xml file is
>> loaded as seed data, but it contains both seed AND demo
>> data: security permissions are seed data, assigning those
>> permissions to demo security groups is demo data. The
>> AccountingTypeData.xml file is loaded as seed data, but it
>> also contains both seed and demo data. Those are just two
>> examples.
>> >
>> > I can commit some of my work as an example of what
>> changes need to be made.
>> >
>> > What do you think?
>> >
>> > -Adrian
>> >
>>
>>
>
>
>
>

Mridul Pathak-2

Re: Discussion: Seed and Demo Data Files

In reply to this post by Adrian Crum-2

+1 for your suggestion.

Though I have one concern here, currently names of data files suggests
the kind of data they are holding. For example,
AccountingTypeData.xml suggests that it has Type Data(including
Enumeration), or DemoFinAccount.xml suggests it has demo data for
FinAccount. This separation is nice. Also if we move all the seed
data for a component to one specific file, it will grow too large
affecting readability, and file for Demo Data can be much more
larger. May be we can use the following format:
[Component-Name][Kind-of-Data(optional)][reader-name]Data.xml.

--
Thanks,
Mridul Pathak
HotWax Media Pvt. Ltd.
http://www.hotwaxmedia.com

On 05-Jun-09, at 12:59 PM, Adrian Crum wrote:

>
> I was about to commit the change in the Asset Maintenance component
> (that our company uses OOTB) - but I realized that the change would
> have to include framework component changes. I ended up dropping the
> effort.
>
> The bottom line is - it's an issue that has to be addressed (similar
> to the BigDecimal or the HTTP security issues we recently addressed)
> - but I'm not going to push it unless others see the benefit.
>
> -Adrian
>
>
> --- On Fri, 6/5/09, Jacques Le Roux <[hidden email]>
> wrote:
>
>> From: Jacques Le Roux <[hidden email]>
>> Subject: Re: Discussion: Seed and Demo Data Files
>> To: [hidden email]
>> Date: Friday, June 5, 2009, 12:06 AM
>> +1 for differentiating data files
>> names
>> +1 for good examples commited (or if easier shown
>> somewhere)
>>
>> Jacques
>>
>> From: "Adrian Crum" <[hidden email]>
>>> I'm getting ready to deploy the latest trunk to our
>> production server, and I have encountered a few problems
>> with the way the seed data is set up.
>>>
>>> In order to get the latest features to work with my
>> existing database, I have to run ant run-install-seed. The
>> problem is, there is a lot of demo data in the seed files.
>> And I really mean a LOT. After running run-install-seed, my
>> database contains a lot of demo data that I had previously
>> removed.
>>>
>>> I would like to suggest the following best practices:
>>>
>>> 1. Keep seed-initial, seed, and demo data in separate
>> XML files.
>>> 2. Name the files accordingly, so their purpose is
>> clear. Example: [component name]InitialData.xml, [component
>> name]SeedData.xml, and [component name]DemoData.xml.
>>>
>>> Right now I'm working on sorting out the data for the
>> accounting component. The AccountingSecurityData.xml file is
>> loaded as seed data, but it contains both seed AND demo
>> data: security permissions are seed data, assigning those
>> permissions to demo security groups is demo data. The
>> AccountingTypeData.xml file is loaded as seed data, but it
>> also contains both seed and demo data. Those are just two
>> examples.
>>>
>>> I can commit some of my work as an example of what
>> changes need to be made.
>>>
>>> What do you think?
>>>
>>> -Adrian
>>>
>>
>>
>
>
>

smime.p7s (2K) Download Attachment

David E Jones-3

Re: Discussion: Seed and Demo Data Files

In reply to this post by Adrian Crum

I haven't really reviewed this for a while, and it's really
frustrating to hear this assessment.

This is definitely a critical effort, especially for those who are
maintaining production instances and need to update seed data every
time they update code. It's a probably we (hopefully!) all have.

As for the file naming conventions, to keep it simple we should
probably just say "[descriptive prefix]" instead of "[component-
name]", kind of along the lines of what Mridul mentioned.

On a related note, the main documentation for this is in the build.xml
file (in the ofbiz-home directory)... and I've been thinking that
maybe we should have something in the Best Practices Guide, or in a
sub-page of that. Maybe then more people would find it and use it.

-David

On Jun 4, 2009, at 11:11 AM, Adrian Crum wrote:

Adrian Crum

Re: Discussion: Seed and Demo Data Files

Sharing some more thoughts on this...

I tried doing a test run of installing seed-initial and seed data only.
I couldn't get logged in because the admin user login is in demo data -
not seed. I moved it to seed-initial - since the account should only be
created/updated during the initial install (from my perspective anyway).
I still couldn't log in because all of the components put their admin
permissions in the FULLADMIN security group, and I had left the security
group assignments in demo data. So, that security group would have to be
part of seed data too.

The goal I was aiming for was to set up a trial new installation that
had only the data needed to get the system running. It ended up being a
bigger task than I anticipated, so I dropped it.

The bottom line is, I started out thinking the boundary between seed
data and demo data is clear-cut and it would be easy to separate the
two, but it isn't.

-Adrian

David E Jones wrote:

>
> I haven't really reviewed this for a while, and it's really frustrating
> to hear this assessment.
>
> This is definitely a critical effort, especially for those who are
> maintaining production instances and need to update seed data every time
> they update code. It's a probably we (hopefully!) all have.
>
> As for the file naming conventions, to keep it simple we should probably
> just say "[descriptive prefix]" instead of "[component-name]", kind of
> along the lines of what Mridul mentioned.
>
> On a related note, the main documentation for this is in the build.xml
> file (in the ofbiz-home directory)... and I've been thinking that maybe
> we should have something in the Best Practices Guide, or in a sub-page
> of that. Maybe then more people would find it and use it.
>
> -David
>
>
> On Jun 4, 2009, at 11:11 AM, Adrian Crum wrote:
>
>> I'm getting ready to deploy the latest trunk to our production server,
>> and I have encountered a few problems with the way the seed data is
>> set up.
>>
>> In order to get the latest features to work with my existing database,
>> I have to run ant run-install-seed. The problem is, there is a lot of
>> demo data in the seed files. And I really mean a LOT. After running
>> run-install-seed, my database contains a lot of demo data that I had
>> previously removed.
>>
>> I would like to suggest the following best practices:
>>
>> 1. Keep seed-initial, seed, and demo data in separate XML files.
>> 2. Name the files accordingly, so their purpose is clear. Example:
>> [component name]InitialData.xml, [component name]SeedData.xml, and
>> [component name]DemoData.xml.
>>
>> Right now I'm working on sorting out the data for the accounting
>> component. The AccountingSecurityData.xml file is loaded as seed data,
>> but it contains both seed AND demo data: security permissions are seed
>> data, assigning those permissions to demo security groups is demo
>> data. The AccountingTypeData.xml file is loaded as seed data, but it
>> also contains both seed and demo data. Those are just two examples.
>>
>> I can commit some of my work as an example of what changes need to be
>> made.
>>
>> What do you think?
>>
>> -Adrian
>
>

Adam Heath-2

Re: Discussion: Seed and Demo Data Files

Adrian Crum wrote:

> Sharing some more thoughts on this...
>
> I tried doing a test run of installing seed-initial and seed data only.
> I couldn't get logged in because the admin user login is in demo data -
> not seed. I moved it to seed-initial - since the account should only be
> created/updated during the initial install (from my perspective anyway).
> I still couldn't log in because all of the components put their admin
> permissions in the FULLADMIN security group, and I had left the security
> group assignments in demo data. So, that security group would have to be
> part of seed data too.

No.

There's an ant target you are supposed to call that created the admin
user login.

David E Jones-3

Re: Discussion: Seed and Demo Data Files

On Jun 5, 2009, at 11:03 AM, Adam Heath wrote:

> Adrian Crum wrote:
>> Sharing some more thoughts on this...
>>
>> I tried doing a test run of installing seed-initial and seed data
>> only.
>> I couldn't get logged in because the admin user login is in demo
>> data -
>> not seed. I moved it to seed-initial - since the account should
>> only be
>> created/updated during the initial install (from my perspective
>> anyway).
>> I still couldn't log in because all of the components put their admin
>> permissions in the FULLADMIN security group, and I had left the
>> security
>> group assignments in demo data. So, that security group would have
>> to be
>> part of seed data too.
>
> No.
>
> There's an ant target you are supposed to call that created the admin
> user login.

Yes, this is a good point.

Don't confuse the term "seed" with the system being all configured and
ready to run, or implying that any part of it should be expected to
run as desired (including login).

The definitions in the entityengine.xml file where these are defined
might be helpful:

seed = OFBiz and External Seed Data - to be maintained along
with source and updated whenever a system deployment is updated
seed-initial = OFBiz and External Seed Data - to be maintained along
with source like other seed data, but only loaded initially and not
updated when a system is updated except manually reviewing each line
demo = OFBiz Only Demo Data
ext = External General Data (custom)
ext-test = External Test Data (custom)
ext-demo = External Demo Data (custom)

The admin user is, IMO (and based on discussions a bit ago), is part
of demo data and not seed or seed-initial.

Based on that, Adrian do you still think there is a big issue with
data that is in seed versus demo data files? I'm guessing there are
still some small issues with that, but hopefully it's not as big a
deal as you thought before...

-David

Jacopo Cappellato-4

Re: Discussion: Seed and Demo Data Files

In reply to this post by Adam Heath-2

On Jun 5, 2009, at 7:03 PM, Adam Heath wrote:

That is correct; Adrian, please refer to this quick notes:

http://docs.ofbiz.org/display/~jacopoc/Quick+Start+Guide+1+-+Getting
+and+Running+OFBiz+in+5+steps

this is an extract from that page:

"If you don't want to load the demo data but only the seed data, use
the following command:
ant run-install-extseed create-admin-user-login
and when prompted, enter a user login id that will be created for you
(the initial temporary password is "ofbiz"). Or you can also pass the
user login id as a parameters with the following command:

ant run-install-extseed load-admin-user-login -DuserLoginId=<ENTER-A-
USER-LOGIN-HERE>"

Hope it helps,

Jacopo

smime.p7s (3K) Download Attachment

Adrian Crum-2

Re: Discussion: Seed and Demo Data Files

In reply to this post by Adrian Crum

Adam and David,

Thank you very much for your replies - they were very helpful. Yes, there is still an issue with demo data mixed in with seed data. I'll look into it again armed with the information I just gained.

-Adrian

--- On Fri, 6/5/09, David E Jones <[hidden email]> wrote:

> From: David E Jones <[hidden email]>
> Subject: Re: Discussion: Seed and Demo Data Files
> To: [hidden email]
> Date: Friday, June 5, 2009, 10:50 AM
>
> On Jun 5, 2009, at 11:03 AM, Adam Heath wrote:
>
> > Adrian Crum wrote:
> >> Sharing some more thoughts on this...
> >>
> >> I tried doing a test run of installing
> seed-initial and seed data only.
> >> I couldn't get logged in because the admin user
> login is in demo data -
> >> not seed. I moved it to seed-initial - since the
> account should only be
> >> created/updated during the initial install (from
> my perspective anyway).
> >> I still couldn't log in because all of the
> components put their admin
> >> permissions in the FULLADMIN security group, and I
> had left the security
> >> group assignments in demo data. So, that security
> group would have to be
> >> part of seed data too.
> >
> > No.
> >
> > There's an ant target you are supposed to call that
> created the admin
> > user login.
>
>
> Yes, this is a good point.
>
> Don't confuse the term "seed" with the system being all
> configured and ready to run, or implying that any part of it
> should be expected to run as desired (including login).
>
> The definitions in the entityengine.xml file where these
> are defined might be helpful:
>
> seed    = OFBiz and
> External Seed Data - to be maintained along with source and
> updated whenever a system deployment is updated
> seed-initial = OFBiz and External Seed Data - to be
> maintained along with source like other seed data, but only
> loaded initially and not updated when a system is updated
> except manually reviewing each line
> demo    = OFBiz Only
> Demo Data
> ext = External General
> Data (custom)
> ext-test    = External Test Data
> (custom)
> ext-demo    = External Demo Data
> (custom)
>
> The admin user is, IMO (and based on discussions a bit
> ago), is part of demo data and not seed or seed-initial.
>
> Based on that, Adrian do you still think there is a big
> issue with data that is in seed versus demo data files? I'm
> guessing there are still some small issues with that, but
> hopefully it's not as big a deal as you thought before...
>
> -David
>
>

BJ Freeman

Re: Discussion: Seed and Demo Data Files

In reply to this post by Adam Heath-2

I just did an install using your 5 steps.
here is the results.
the Themes are not loaded
The backend defaults to bluelight style
however there is no way to select any styles.

other than those, it went great.

Jacopo Cappellato sent the following on 6/6/2009 1:47 AM:

> On Jun 5, 2009, at 7:03 PM, Adam Heath wrote:
>
>> Adrian Crum wrote:
>>> Sharing some more thoughts on this...
>>>
>>> I tried doing a test run of installing seed-initial and seed data only.
>>> I couldn't get logged in because the admin user login is in demo data -
>>> not seed. I moved it to seed-initial - since the account should only be
>>> created/updated during the initial install (from my perspective anyway).
>>> I still couldn't log in because all of the components put their admin
>>> permissions in the FULLADMIN security group, and I had left the security
>>> group assignments in demo data. So, that security group would have to be
>>> part of seed data too.
>>
>> No.
>>
>> There's an ant target you are supposed to call that created the admin
>> user login.
>>
>
> That is correct; Adrian, please refer to this quick notes:
>
> http://docs.ofbiz.org/display/~jacopoc/Quick+Start+Guide+1+-+Getting+and+Running+OFBiz+in+5+steps
>
>
> this is an extract from that page:
>
> "If you don't want to load the demo data but only the seed data, use the
> following command:
> ant run-install-extseed create-admin-user-login
> and when prompted, enter a user login id that will be created for you
> (the initial temporary password is "ofbiz"). Or you can also pass the
> user login id as a parameters with the following command:
>
> ant run-install-extseed load-admin-user-login
> -DuserLoginId=<ENTER-A-USER-LOGIN-HERE>"
>
> Hope it helps,
>
> Jacopo

--
BJ Freeman
http://www.businessesnetwork.com/automation
http://bjfreeman.elance.com
http://www.linkedin.com/profile?viewProfile=&key=1237480&locale=en_US&trk=tab_pro
Systems Integrator.