[Proposal] moving all data in applications to the data-model component

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

[Proposal] moving all data in applications to the data-model component

taher
 Hello Everyone,

Data loading in OFBiz is quite fragile. A small change could crash the
system and finding out the reason can be difficult. I really felt the
pain of this when trying to disentangle the framework from data
dependencies on plugins [1].

I've been thinking about a solution to reduce fragility, and here are
my thoughts:
- The less data files the better.
- The less data the better.
- The less distance (e.g. same directory) between data files the better.
- No circular dependencies (file dependencies should be like a tree).
- Fail fast [2].

Therefore, I suggest the following action points:
- Create a directory structure similar to [3] in the datamodel component.
- For any data that two or more files depend on, immediately move it
to the common-<reader>.xml file.
- For any data that does not belong to any domain (e.g. marketing,
content, etc ..) move it to the common-<reader>.xml.
- Maintain a strict dependency hierarchy of demo -> seed -> seed-initial.
- Gradually and carefully move data from all applications to the
datamodel component as per above guidelines.
- Cleanup and refactor the data as we move it including any tests that
depend on it.

What do you think of this proposal? Does it sound reasonable? Also
given the size of what I'm proposing I think we need multiple people
to work on this. So is anyone interested in taking part in this
initiative?

Cheers,

Taher Alkhateeb

[1] https://issues.apache.org/jira/browse/OFBIZ-9322
[2] https://lists.apache.org/thread.html/a18b5d4086a07ff34a0cb950999e8f8e869a6aa7efa2fc866a75a7b6@%3Cdev.ofbiz.apache.org%3E
[3] directory structure:
data/
├── demo
│   ├── accounting-demo.xml
│   ├── common-demo.xml
│   ├── content-demo.xml
│   ├── humanres-demo.xml
│   ├── manufacturing-demo.xml
│   ├── marketing-demo.xml
│   ├── order-demo.xml
│   ├── party-demo.xml
│   ├── product-demo.xml
│   ├── shipment-demo.xml
│   └── workeffort-demo.xml
├── seed
│   ├── accounting-seed.xml
│   ├── common-seed.xml
│   ├── content-seed.xml
│   ├── humanres-seed.xml
│   ├── manufacturing-seed.xml
│   ├── marketing-seed.xml
│   ├── order-seed.xml
│   ├── party-seed.xml
│   ├── product-seed.xml
│   ├── shipment-seed.xml
│   └── workeffort-seed.xml
└── seed-initial
    ├── accounting-seed-initial.xml
    ├── common-seed-initial.xml
    ├── content-seed-initial.xml
    ├── humanres-seed-initial.xml
    ├── manufacturing-seed-initial.xml
    ├── marketing-seed-initial.xml
    ├── order-seed-initial.xml
    ├── party-seed-initial.xml
    ├── product-seed-initial.xml
    ├── shipment-seed-initial.xml
    └── workeffort-seed-initial.xml
Reply | Threaded
Open this post in threaded view
|

Re: [Proposal] moving all data in applications to the data-model component

Paul Foxworthy
Hi Taher,

Makes sense to me.

Would any plugin defining custom entities also have a similar /data tree?

Cheers

Paul


On 14 July 2017 at 19:59, Taher Alkhateeb <[hidden email]>
wrote:

>  Hello Everyone,
>
> Data loading in OFBiz is quite fragile. A small change could crash the
> system and finding out the reason can be difficult. I really felt the
> pain of this when trying to disentangle the framework from data
> dependencies on plugins [1].
>
> I've been thinking about a solution to reduce fragility, and here are
> my thoughts:
> - The less data files the better.
> - The less data the better.
> - The less distance (e.g. same directory) between data files the better.
> - No circular dependencies (file dependencies should be like a tree).
> - Fail fast [2].
>
> Therefore, I suggest the following action points:
> - Create a directory structure similar to [3] in the datamodel component.
> - For any data that two or more files depend on, immediately move it
> to the common-<reader>.xml file.
> - For any data that does not belong to any domain (e.g. marketing,
> content, etc ..) move it to the common-<reader>.xml.
> - Maintain a strict dependency hierarchy of demo -> seed -> seed-initial.
> - Gradually and carefully move data from all applications to the
> datamodel component as per above guidelines.
> - Cleanup and refactor the data as we move it including any tests that
> depend on it.
>
> What do you think of this proposal? Does it sound reasonable? Also
> given the size of what I'm proposing I think we need multiple people
> to work on this. So is anyone interested in taking part in this
> initiative?
>
> Cheers,
>
> Taher Alkhateeb
>
> [1] https://issues.apache.org/jira/browse/OFBIZ-9322
> [2] https://lists.apache.org/thread.html/a18b5d4086a07ff34a0cb950999e8f
> 8e869a6aa7efa2fc866a75a7b6@%3Cdev.ofbiz.apache.org%3E
> [3] directory structure:
> data/
> ├── demo
> │   ├── accounting-demo.xml
> │   ├── common-demo.xml
> │   ├── content-demo.xml
> │   ├── humanres-demo.xml
> │   ├── manufacturing-demo.xml
> │   ├── marketing-demo.xml
> │   ├── order-demo.xml
> │   ├── party-demo.xml
> │   ├── product-demo.xml
> │   ├── shipment-demo.xml
> │   └── workeffort-demo.xml
> ├── seed
> │   ├── accounting-seed.xml
> │   ├── common-seed.xml
> │   ├── content-seed.xml
> │   ├── humanres-seed.xml
> │   ├── manufacturing-seed.xml
> │   ├── marketing-seed.xml
> │   ├── order-seed.xml
> │   ├── party-seed.xml
> │   ├── product-seed.xml
> │   ├── shipment-seed.xml
> │   └── workeffort-seed.xml
> └── seed-initial
>     ├── accounting-seed-initial.xml
>     ├── common-seed-initial.xml
>     ├── content-seed-initial.xml
>     ├── humanres-seed-initial.xml
>     ├── manufacturing-seed-initial.xml
>     ├── marketing-seed-initial.xml
>     ├── order-seed-initial.xml
>     ├── party-seed-initial.xml
>     ├── product-seed-initial.xml
>     ├── shipment-seed-initial.xml
>     └── workeffort-seed-initial.xml
>



--
Coherent Software Australia Pty Ltd
PO Box 2773
Cheltenham Vic 3192
Australia

Phone: +61 3 9585 6788
Web: http://www.coherentsoftware.com.au/
Email: [hidden email]
--
Coherent Software Australia Pty Ltd
http://www.coherentsoftware.com.au/

Bonsai ERP, the all-inclusive ERP system
http://www.bonsaierp.com.au/
Reply | Threaded
Open this post in threaded view
|

Re: [Proposal] moving all data in applications to the data-model component

Jacopo Cappellato-5
In reply to this post by taher
+1 to everything you wrote. I will do my best to help with the initiative.

Thank you Taher.

Jacopo

On Fri, Jul 14, 2017 at 11:59 AM, Taher Alkhateeb <
[hidden email]> wrote:

>  Hello Everyone,
>
> Data loading in OFBiz is quite fragile. A small change could crash the
> system and finding out the reason can be difficult. I really felt the
> pain of this when trying to disentangle the framework from data
> dependencies on plugins [1].
>
> I've been thinking about a solution to reduce fragility, and here are
> my thoughts:
> - The less data files the better.
> - The less data the better.
> - The less distance (e.g. same directory) between data files the better.
> - No circular dependencies (file dependencies should be like a tree).
> - Fail fast [2].
>
> Therefore, I suggest the following action points:
> - Create a directory structure similar to [3] in the datamodel component.
> - For any data that two or more files depend on, immediately move it
> to the common-<reader>.xml file.
> - For any data that does not belong to any domain (e.g. marketing,
> content, etc ..) move it to the common-<reader>.xml.
> - Maintain a strict dependency hierarchy of demo -> seed -> seed-initial.
> - Gradually and carefully move data from all applications to the
> datamodel component as per above guidelines.
> - Cleanup and refactor the data as we move it including any tests that
> depend on it.
>
> What do you think of this proposal? Does it sound reasonable? Also
> given the size of what I'm proposing I think we need multiple people
> to work on this. So is anyone interested in taking part in this
> initiative?
>
> Cheers,
>
> Taher Alkhateeb
>
> [1] https://issues.apache.org/jira/browse/OFBIZ-9322
> [2] https://lists.apache.org/thread.html/a18b5d4086a07ff34a0cb950999e8f
> 8e869a6aa7efa2fc866a75a7b6@%3Cdev.ofbiz.apache.org%3E
> [3] directory structure:
> data/
> ├── demo
> │   ├── accounting-demo.xml
> │   ├── common-demo.xml
> │   ├── content-demo.xml
> │   ├── humanres-demo.xml
> │   ├── manufacturing-demo.xml
> │   ├── marketing-demo.xml
> │   ├── order-demo.xml
> │   ├── party-demo.xml
> │   ├── product-demo.xml
> │   ├── shipment-demo.xml
> │   └── workeffort-demo.xml
> ├── seed
> │   ├── accounting-seed.xml
> │   ├── common-seed.xml
> │   ├── content-seed.xml
> │   ├── humanres-seed.xml
> │   ├── manufacturing-seed.xml
> │   ├── marketing-seed.xml
> │   ├── order-seed.xml
> │   ├── party-seed.xml
> │   ├── product-seed.xml
> │   ├── shipment-seed.xml
> │   └── workeffort-seed.xml
> └── seed-initial
>     ├── accounting-seed-initial.xml
>     ├── common-seed-initial.xml
>     ├── content-seed-initial.xml
>     ├── humanres-seed-initial.xml
>     ├── manufacturing-seed-initial.xml
>     ├── marketing-seed-initial.xml
>     ├── order-seed-initial.xml
>     ├── party-seed-initial.xml
>     ├── product-seed-initial.xml
>     ├── shipment-seed-initial.xml
>     └── workeffort-seed-initial.xml
>
Reply | Threaded
Open this post in threaded view
|

Re: [Proposal] moving all data in applications to the data-model component

taher
In reply to this post by Paul Foxworthy
Hi Paul,

I don't have an opinion on plugins so far as I didn't think enough
about it. I just thought since we centralized the entities in
"data-model" we might as well centralize the data itself. Hopefully
we'll discuss architectural proposals for plugins at some point but
I'm trying to focus on cleaning stuff from the inside out.

With that being said, I always welcome ideas to enrich this
discussion, and I hope more people find this thread interesting to
pitch in.

On Mon, Jul 17, 2017 at 9:00 AM, Paul Foxworthy <[hidden email]> wrote:

> Hi Taher,
>
> Makes sense to me.
>
> Would any plugin defining custom entities also have a similar /data tree?
>
> Cheers
>
> Paul
>
>
> On 14 July 2017 at 19:59, Taher Alkhateeb <[hidden email]>
> wrote:
>
>>  Hello Everyone,
>>
>> Data loading in OFBiz is quite fragile. A small change could crash the
>> system and finding out the reason can be difficult. I really felt the
>> pain of this when trying to disentangle the framework from data
>> dependencies on plugins [1].
>>
>> I've been thinking about a solution to reduce fragility, and here are
>> my thoughts:
>> - The less data files the better.
>> - The less data the better.
>> - The less distance (e.g. same directory) between data files the better.
>> - No circular dependencies (file dependencies should be like a tree).
>> - Fail fast [2].
>>
>> Therefore, I suggest the following action points:
>> - Create a directory structure similar to [3] in the datamodel component.
>> - For any data that two or more files depend on, immediately move it
>> to the common-<reader>.xml file.
>> - For any data that does not belong to any domain (e.g. marketing,
>> content, etc ..) move it to the common-<reader>.xml.
>> - Maintain a strict dependency hierarchy of demo -> seed -> seed-initial.
>> - Gradually and carefully move data from all applications to the
>> datamodel component as per above guidelines.
>> - Cleanup and refactor the data as we move it including any tests that
>> depend on it.
>>
>> What do you think of this proposal? Does it sound reasonable? Also
>> given the size of what I'm proposing I think we need multiple people
>> to work on this. So is anyone interested in taking part in this
>> initiative?
>>
>> Cheers,
>>
>> Taher Alkhateeb
>>
>> [1] https://issues.apache.org/jira/browse/OFBIZ-9322
>> [2] https://lists.apache.org/thread.html/a18b5d4086a07ff34a0cb950999e8f
>> 8e869a6aa7efa2fc866a75a7b6@%3Cdev.ofbiz.apache.org%3E
>> [3] directory structure:
>> data/
>> ├── demo
>> │   ├── accounting-demo.xml
>> │   ├── common-demo.xml
>> │   ├── content-demo.xml
>> │   ├── humanres-demo.xml
>> │   ├── manufacturing-demo.xml
>> │   ├── marketing-demo.xml
>> │   ├── order-demo.xml
>> │   ├── party-demo.xml
>> │   ├── product-demo.xml
>> │   ├── shipment-demo.xml
>> │   └── workeffort-demo.xml
>> ├── seed
>> │   ├── accounting-seed.xml
>> │   ├── common-seed.xml
>> │   ├── content-seed.xml
>> │   ├── humanres-seed.xml
>> │   ├── manufacturing-seed.xml
>> │   ├── marketing-seed.xml
>> │   ├── order-seed.xml
>> │   ├── party-seed.xml
>> │   ├── product-seed.xml
>> │   ├── shipment-seed.xml
>> │   └── workeffort-seed.xml
>> └── seed-initial
>>     ├── accounting-seed-initial.xml
>>     ├── common-seed-initial.xml
>>     ├── content-seed-initial.xml
>>     ├── humanres-seed-initial.xml
>>     ├── manufacturing-seed-initial.xml
>>     ├── marketing-seed-initial.xml
>>     ├── order-seed-initial.xml
>>     ├── party-seed-initial.xml
>>     ├── product-seed-initial.xml
>>     ├── shipment-seed-initial.xml
>>     └── workeffort-seed-initial.xml
>>
>
>
>
> --
> Coherent Software Australia Pty Ltd
> PO Box 2773
> Cheltenham Vic 3192
> Australia
>
> Phone: +61 3 9585 6788
> Web: http://www.coherentsoftware.com.au/
> Email: [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [Proposal] moving all data in applications to the data-model component

Michael Brohl-3
In reply to this post by taher
+1

Michael

Am 14.07.17 um 11:59 schrieb Taher Alkhateeb:

>   Hello Everyone,
>
> Data loading in OFBiz is quite fragile. A small change could crash the
> system and finding out the reason can be difficult. I really felt the
> pain of this when trying to disentangle the framework from data
> dependencies on plugins [1].
>
> I've been thinking about a solution to reduce fragility, and here are
> my thoughts:
> - The less data files the better.
> - The less data the better.
> - The less distance (e.g. same directory) between data files the better.
> - No circular dependencies (file dependencies should be like a tree).
> - Fail fast [2].
>
> Therefore, I suggest the following action points:
> - Create a directory structure similar to [3] in the datamodel component.
> - For any data that two or more files depend on, immediately move it
> to the common-<reader>.xml file.
> - For any data that does not belong to any domain (e.g. marketing,
> content, etc ..) move it to the common-<reader>.xml.
> - Maintain a strict dependency hierarchy of demo -> seed -> seed-initial.
> - Gradually and carefully move data from all applications to the
> datamodel component as per above guidelines.
> - Cleanup and refactor the data as we move it including any tests that
> depend on it.
>
> What do you think of this proposal? Does it sound reasonable? Also
> given the size of what I'm proposing I think we need multiple people
> to work on this. So is anyone interested in taking part in this
> initiative?
>
> Cheers,
>
> Taher Alkhateeb
>
> [1] https://issues.apache.org/jira/browse/OFBIZ-9322
> [2] https://lists.apache.org/thread.html/a18b5d4086a07ff34a0cb950999e8f8e869a6aa7efa2fc866a75a7b6@%3Cdev.ofbiz.apache.org%3E
> [3] directory structure:
> data/
> ├── demo
> │   ├── accounting-demo.xml
> │   ├── common-demo.xml
> │   ├── content-demo.xml
> │   ├── humanres-demo.xml
> │   ├── manufacturing-demo.xml
> │   ├── marketing-demo.xml
> │   ├── order-demo.xml
> │   ├── party-demo.xml
> │   ├── product-demo.xml
> │   ├── shipment-demo.xml
> │   └── workeffort-demo.xml
> ├── seed
> │   ├── accounting-seed.xml
> │   ├── common-seed.xml
> │   ├── content-seed.xml
> │   ├── humanres-seed.xml
> │   ├── manufacturing-seed.xml
> │   ├── marketing-seed.xml
> │   ├── order-seed.xml
> │   ├── party-seed.xml
> │   ├── product-seed.xml
> │   ├── shipment-seed.xml
> │   └── workeffort-seed.xml
> └── seed-initial
>      ├── accounting-seed-initial.xml
>      ├── common-seed-initial.xml
>      ├── content-seed-initial.xml
>      ├── humanres-seed-initial.xml
>      ├── manufacturing-seed-initial.xml
>      ├── marketing-seed-initial.xml
>      ├── order-seed-initial.xml
>      ├── party-seed-initial.xml
>      ├── product-seed-initial.xml
>      ├── shipment-seed-initial.xml
>      └── workeffort-seed-initial.xml


smime.p7s (5K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [Proposal] moving all data in applications to the data-model component

James Yong-2
In reply to this post by taher
+1

Regards,
James Yong

On 2017-07-14 17:59 (+0800), Taher Alkhateeb <[hidden email]> wrote:

>  Hello Everyone,
>
> Data loading in OFBiz is quite fragile. A small change could crash the
> system and finding out the reason can be difficult. I really felt the
> pain of this when trying to disentangle the framework from data
> dependencies on plugins [1].
>
> I've been thinking about a solution to reduce fragility, and here are
> my thoughts:
> - The less data files the better.
> - The less data the better.
> - The less distance (e.g. same directory) between data files the better.
> - No circular dependencies (file dependencies should be like a tree).
> - Fail fast [2].
>
> Therefore, I suggest the following action points:
> - Create a directory structure similar to [3] in the datamodel component.
> - For any data that two or more files depend on, immediately move it
> to the common-<reader>.xml file.
> - For any data that does not belong to any domain (e.g. marketing,
> content, etc ..) move it to the common-<reader>.xml.
> - Maintain a strict dependency hierarchy of demo -> seed -> seed-initial.
> - Gradually and carefully move data from all applications to the
> datamodel component as per above guidelines.
> - Cleanup and refactor the data as we move it including any tests that
> depend on it.
>
> What do you think of this proposal? Does it sound reasonable? Also
> given the size of what I'm proposing I think we need multiple people
> to work on this. So is anyone interested in taking part in this
> initiative?
>
> Cheers,
>
> Taher Alkhateeb
>
> [1] https://issues.apache.org/jira/browse/OFBIZ-9322
> [2] https://lists.apache.org/thread.html/a18b5d4086a07ff34a0cb950999e8f8e869a6aa7efa2fc866a75a7b6@%3Cdev.ofbiz.apache.org%3E
> [3] directory structure:
> data/
> ├── demo
> │   ├── accounting-demo.xml
> │   ├── common-demo.xml
> │   ├── content-demo.xml
> │   ├── humanres-demo.xml
> │   ├── manufacturing-demo.xml
> │   ├── marketing-demo.xml
> │   ├── order-demo.xml
> │   ├── party-demo.xml
> │   ├── product-demo.xml
> │   ├── shipment-demo.xml
> │   └── workeffort-demo.xml
> ├── seed
> │   ├── accounting-seed.xml
> │   ├── common-seed.xml
> │   ├── content-seed.xml
> │   ├── humanres-seed.xml
> │   ├── manufacturing-seed.xml
> │   ├── marketing-seed.xml
> │   ├── order-seed.xml
> │   ├── party-seed.xml
> │   ├── product-seed.xml
> │   ├── shipment-seed.xml
> │   └── workeffort-seed.xml
> └── seed-initial
>     ├── accounting-seed-initial.xml
>     ├── common-seed-initial.xml
>     ├── content-seed-initial.xml
>     ├── humanres-seed-initial.xml
>     ├── manufacturing-seed-initial.xml
>     ├── marketing-seed-initial.xml
>     ├── order-seed-initial.xml
>     ├── party-seed-initial.xml
>     ├── product-seed-initial.xml
>     ├── shipment-seed-initial.xml
>     └── workeffort-seed-initial.xml
>
Reply | Threaded
Open this post in threaded view
|

SV: [Proposal] moving all data in applications to the data-model component

Sakke Ferraris
Hi,

I just joined, and I'd like to help on this.

BR, Sakke Ferraris

-----Ursprungligt meddelande-----
Från: James Yong [mailto:[hidden email]]
Skickat: den 18 juli 2017 10:45
Till: [hidden email]
Ämne: Re: [Proposal] moving all data in applications to the data-model component

+1

Regards,
James Yong

On 2017-07-14 17:59 (+0800), Taher Alkhateeb <[hidden email]> wrote:

>  Hello Everyone,
>
> Data loading in OFBiz is quite fragile. A small change could crash the
> system and finding out the reason can be difficult. I really felt the
> pain of this when trying to disentangle the framework from data
> dependencies on plugins [1].
>
> I've been thinking about a solution to reduce fragility, and here are
> my thoughts:
> - The less data files the better.
> - The less data the better.
> - The less distance (e.g. same directory) between data files the better.
> - No circular dependencies (file dependencies should be like a tree).
> - Fail fast [2].
>
> Therefore, I suggest the following action points:
> - Create a directory structure similar to [3] in the datamodel component.
> - For any data that two or more files depend on, immediately move it
> to the common-<reader>.xml file.
> - For any data that does not belong to any domain (e.g. marketing,
> content, etc ..) move it to the common-<reader>.xml.
> - Maintain a strict dependency hierarchy of demo -> seed -> seed-initial.
> - Gradually and carefully move data from all applications to the
> datamodel component as per above guidelines.
> - Cleanup and refactor the data as we move it including any tests that
> depend on it.
>
> What do you think of this proposal? Does it sound reasonable? Also
> given the size of what I'm proposing I think we need multiple people
> to work on this. So is anyone interested in taking part in this
> initiative?
>
> Cheers,
>
> Taher Alkhateeb
>
> [1] https://issues.apache.org/jira/browse/OFBIZ-9322
> [2]
> https://lists.apache.org/thread.html/a18b5d4086a07ff34a0cb950999e8f8e8
> 69a6aa7efa2fc866a75a7b6@%3Cdev.ofbiz.apache.org%3E
> [3] directory structure:
> data/
> ├── demo
> │   ├── accounting-demo.xml
> │   ├── common-demo.xml
> │   ├── content-demo.xml
> │   ├── humanres-demo.xml
> │   ├── manufacturing-demo.xml
> │   ├── marketing-demo.xml
> │   ├── order-demo.xml
> │   ├── party-demo.xml
> │   ├── product-demo.xml
> │   ├── shipment-demo.xml
> │   └── workeffort-demo.xml
> ├── seed
> │   ├── accounting-seed.xml
> │   ├── common-seed.xml
> │   ├── content-seed.xml
> │   ├── humanres-seed.xml
> │   ├── manufacturing-seed.xml
> │   ├── marketing-seed.xml
> │   ├── order-seed.xml
> │   ├── party-seed.xml
> │   ├── product-seed.xml
> │   ├── shipment-seed.xml
> │   └── workeffort-seed.xml
> └── seed-initial
>     ├── accounting-seed-initial.xml
>     ├── common-seed-initial.xml
>     ├── content-seed-initial.xml
>     ├── humanres-seed-initial.xml
>     ├── manufacturing-seed-initial.xml
>     ├── marketing-seed-initial.xml
>     ├── order-seed-initial.xml
>     ├── party-seed-initial.xml
>     ├── product-seed-initial.xml
>     ├── shipment-seed-initial.xml
>     └── workeffort-seed-initial.xml
>
Reply | Threaded
Open this post in threaded view
|

Re: [Proposal] moving all data in applications to the data-model component

Deepak Dixit-3
+1 ,

Thanks & Regards
--
Deepak Dixit
www.hotwaxsystems.com
www.hotwax.co

On Tue, Jul 18, 2017 at 2:29 PM, Sakke Ferraris <[hidden email]> wrote:

> Hi,
>
> I just joined, and I'd like to help on this.
>
> BR, Sakke Ferraris
>
> -----Ursprungligt meddelande-----
> Från: James Yong [mailto:[hidden email]]
> Skickat: den 18 juli 2017 10:45
> Till: [hidden email]
> Ämne: Re: [Proposal] moving all data in applications to the data-model
> component
>
> +1
>
> Regards,
> James Yong
>
> On 2017-07-14 17:59 (+0800), Taher Alkhateeb <[hidden email]>
> wrote:
> >  Hello Everyone,
> >
> > Data loading in OFBiz is quite fragile. A small change could crash the
> > system and finding out the reason can be difficult. I really felt the
> > pain of this when trying to disentangle the framework from data
> > dependencies on plugins [1].
> >
> > I've been thinking about a solution to reduce fragility, and here are
> > my thoughts:
> > - The less data files the better.
> > - The less data the better.
> > - The less distance (e.g. same directory) between data files the better.
> > - No circular dependencies (file dependencies should be like a tree).
> > - Fail fast [2].
> >
> > Therefore, I suggest the following action points:
> > - Create a directory structure similar to [3] in the datamodel component.
> > - For any data that two or more files depend on, immediately move it
> > to the common-<reader>.xml file.
> > - For any data that does not belong to any domain (e.g. marketing,
> > content, etc ..) move it to the common-<reader>.xml.
> > - Maintain a strict dependency hierarchy of demo -> seed -> seed-initial.
> > - Gradually and carefully move data from all applications to the
> > datamodel component as per above guidelines.
> > - Cleanup and refactor the data as we move it including any tests that
> > depend on it.
> >
> > What do you think of this proposal? Does it sound reasonable? Also
> > given the size of what I'm proposing I think we need multiple people
> > to work on this. So is anyone interested in taking part in this
> > initiative?
> >
> > Cheers,
> >
> > Taher Alkhateeb
> >
> > [1] https://issues.apache.org/jira/browse/OFBIZ-9322
> > [2]
> > https://lists.apache.org/thread.html/a18b5d4086a07ff34a0cb950999e8f8e8
> > 69a6aa7efa2fc866a75a7b6@%3Cdev.ofbiz.apache.org%3E
> > [3] directory structure:
> > data/
> > ├── demo
> > │   ├── accounting-demo.xml
> > │   ├── common-demo.xml
> > │   ├── content-demo.xml
> > │   ├── humanres-demo.xml
> > │   ├── manufacturing-demo.xml
> > │   ├── marketing-demo.xml
> > │   ├── order-demo.xml
> > │   ├── party-demo.xml
> > │   ├── product-demo.xml
> > │   ├── shipment-demo.xml
> > │   └── workeffort-demo.xml
> > ├── seed
> > │   ├── accounting-seed.xml
> > │   ├── common-seed.xml
> > │   ├── content-seed.xml
> > │   ├── humanres-seed.xml
> > │   ├── manufacturing-seed.xml
> > │   ├── marketing-seed.xml
> > │   ├── order-seed.xml
> > │   ├── party-seed.xml
> > │   ├── product-seed.xml
> > │   ├── shipment-seed.xml
> > │   └── workeffort-seed.xml
> > └── seed-initial
> >     ├── accounting-seed-initial.xml
> >     ├── common-seed-initial.xml
> >     ├── content-seed-initial.xml
> >     ├── humanres-seed-initial.xml
> >     ├── manufacturing-seed-initial.xml
> >     ├── marketing-seed-initial.xml
> >     ├── order-seed-initial.xml
> >     ├── party-seed-initial.xml
> >     ├── product-seed-initial.xml
> >     ├── shipment-seed-initial.xml
> >     └── workeffort-seed-initial.xml
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: [Proposal] moving all data in applications to the data-model component

Devanshu Vyas-2
In reply to this post by taher
+1

Thanks & Regards,
Devanshu Vyas.

On Fri, Jul 14, 2017 at 3:29 PM, Taher Alkhateeb <[hidden email]
> wrote:

>  Hello Everyone,
>
> Data loading in OFBiz is quite fragile. A small change could crash the
> system and finding out the reason can be difficult. I really felt the
> pain of this when trying to disentangle the framework from data
> dependencies on plugins [1].
>
> I've been thinking about a solution to reduce fragility, and here are
> my thoughts:
> - The less data files the better.
> - The less data the better.
> - The less distance (e.g. same directory) between data files the better.
> - No circular dependencies (file dependencies should be like a tree).
> - Fail fast [2].
>
> Therefore, I suggest the following action points:
> - Create a directory structure similar to [3] in the datamodel component.
> - For any data that two or more files depend on, immediately move it
> to the common-<reader>.xml file.
> - For any data that does not belong to any domain (e.g. marketing,
> content, etc ..) move it to the common-<reader>.xml.
> - Maintain a strict dependency hierarchy of demo -> seed -> seed-initial.
> - Gradually and carefully move data from all applications to the
> datamodel component as per above guidelines.
> - Cleanup and refactor the data as we move it including any tests that
> depend on it.
>
> What do you think of this proposal? Does it sound reasonable? Also
> given the size of what I'm proposing I think we need multiple people
> to work on this. So is anyone interested in taking part in this
> initiative?
>
> Cheers,
>
> Taher Alkhateeb
>
> [1] https://issues.apache.org/jira/browse/OFBIZ-9322
> [2] https://lists.apache.org/thread.html/a18b5d4086a07ff34a0cb950999e8f
> 8e869a6aa7efa2fc866a75a7b6@%3Cdev.ofbiz.apache.org%3E
> [3] directory structure:
> data/
> ├── demo
> │   ├── accounting-demo.xml
> │   ├── common-demo.xml
> │   ├── content-demo.xml
> │   ├── humanres-demo.xml
> │   ├── manufacturing-demo.xml
> │   ├── marketing-demo.xml
> │   ├── order-demo.xml
> │   ├── party-demo.xml
> │   ├── product-demo.xml
> │   ├── shipment-demo.xml
> │   └── workeffort-demo.xml
> ├── seed
> │   ├── accounting-seed.xml
> │   ├── common-seed.xml
> │   ├── content-seed.xml
> │   ├── humanres-seed.xml
> │   ├── manufacturing-seed.xml
> │   ├── marketing-seed.xml
> │   ├── order-seed.xml
> │   ├── party-seed.xml
> │   ├── product-seed.xml
> │   ├── shipment-seed.xml
> │   └── workeffort-seed.xml
> └── seed-initial
>     ├── accounting-seed-initial.xml
>     ├── common-seed-initial.xml
>     ├── content-seed-initial.xml
>     ├── humanres-seed-initial.xml
>     ├── manufacturing-seed-initial.xml
>     ├── marketing-seed-initial.xml
>     ├── order-seed-initial.xml
>     ├── party-seed-initial.xml
>     ├── product-seed-initial.xml
>     ├── shipment-seed-initial.xml
>     └── workeffort-seed-initial.xml
>
Reply | Threaded
Open this post in threaded view
|

Re: [Proposal] moving all data in applications to the data-model component

Sharan-F
In reply to this post by taher
+1

Thanks
Sharan

On 14/07/17 11:59, Taher Alkhateeb wrote:

>   Hello Everyone,
>
> Data loading in OFBiz is quite fragile. A small change could crash the
> system and finding out the reason can be difficult. I really felt the
> pain of this when trying to disentangle the framework from data
> dependencies on plugins [1].
>
> I've been thinking about a solution to reduce fragility, and here are
> my thoughts:
> - The less data files the better.
> - The less data the better.
> - The less distance (e.g. same directory) between data files the better.
> - No circular dependencies (file dependencies should be like a tree).
> - Fail fast [2].
>
> Therefore, I suggest the following action points:
> - Create a directory structure similar to [3] in the datamodel component.
> - For any data that two or more files depend on, immediately move it
> to the common-<reader>.xml file.
> - For any data that does not belong to any domain (e.g. marketing,
> content, etc ..) move it to the common-<reader>.xml.
> - Maintain a strict dependency hierarchy of demo -> seed -> seed-initial.
> - Gradually and carefully move data from all applications to the
> datamodel component as per above guidelines.
> - Cleanup and refactor the data as we move it including any tests that
> depend on it.
>
> What do you think of this proposal? Does it sound reasonable? Also
> given the size of what I'm proposing I think we need multiple people
> to work on this. So is anyone interested in taking part in this
> initiative?
>
> Cheers,
>
> Taher Alkhateeb
>
> [1] https://issues.apache.org/jira/browse/OFBIZ-9322
> [2] https://lists.apache.org/thread.html/a18b5d4086a07ff34a0cb950999e8f8e869a6aa7efa2fc866a75a7b6@%3Cdev.ofbiz.apache.org%3E
> [3] directory structure:
> data/
> ├── demo
> │   ├── accounting-demo.xml
> │   ├── common-demo.xml
> │   ├── content-demo.xml
> │   ├── humanres-demo.xml
> │   ├── manufacturing-demo.xml
> │   ├── marketing-demo.xml
> │   ├── order-demo.xml
> │   ├── party-demo.xml
> │   ├── product-demo.xml
> │   ├── shipment-demo.xml
> │   └── workeffort-demo.xml
> ├── seed
> │   ├── accounting-seed.xml
> │   ├── common-seed.xml
> │   ├── content-seed.xml
> │   ├── humanres-seed.xml
> │   ├── manufacturing-seed.xml
> │   ├── marketing-seed.xml
> │   ├── order-seed.xml
> │   ├── party-seed.xml
> │   ├── product-seed.xml
> │   ├── shipment-seed.xml
> │   └── workeffort-seed.xml
> └── seed-initial
>      ├── accounting-seed-initial.xml
>      ├── common-seed-initial.xml
>      ├── content-seed-initial.xml
>      ├── humanres-seed-initial.xml
>      ├── manufacturing-seed-initial.xml
>      ├── marketing-seed-initial.xml
>      ├── order-seed-initial.xml
>      ├── party-seed-initial.xml
>      ├── product-seed-initial.xml
>      ├── shipment-seed-initial.xml
>      └── workeffort-seed-initial.xml

Reply | Threaded
Open this post in threaded view
|

Re: [Proposal] moving all data in applications to the data-model component

Aditya Sharma
+1

Thanks and Regards,
*Aditya Sharma* | Enterprise Software Engineer
HotWax Systems <http://www.hotwaxsystems.com/>

<https://www.linkedin.com/in/aditya-sharma-78291810a/>

On Tue, Jul 18, 2017 at 3:08 PM, Sharan Foga <[hidden email]> wrote:

> +1
>
> Thanks
> Sharan
>
>
> On 14/07/17 11:59, Taher Alkhateeb wrote:
>
>>   Hello Everyone,
>>
>> Data loading in OFBiz is quite fragile. A small change could crash the
>> system and finding out the reason can be difficult. I really felt the
>> pain of this when trying to disentangle the framework from data
>> dependencies on plugins [1].
>>
>> I've been thinking about a solution to reduce fragility, and here are
>> my thoughts:
>> - The less data files the better.
>> - The less data the better.
>> - The less distance (e.g. same directory) between data files the better.
>> - No circular dependencies (file dependencies should be like a tree).
>> - Fail fast [2].
>>
>> Therefore, I suggest the following action points:
>> - Create a directory structure similar to [3] in the datamodel component.
>> - For any data that two or more files depend on, immediately move it
>> to the common-<reader>.xml file.
>> - For any data that does not belong to any domain (e.g. marketing,
>> content, etc ..) move it to the common-<reader>.xml.
>> - Maintain a strict dependency hierarchy of demo -> seed -> seed-initial.
>> - Gradually and carefully move data from all applications to the
>> datamodel component as per above guidelines.
>> - Cleanup and refactor the data as we move it including any tests that
>> depend on it.
>>
>> What do you think of this proposal? Does it sound reasonable? Also
>> given the size of what I'm proposing I think we need multiple people
>> to work on this. So is anyone interested in taking part in this
>> initiative?
>>
>> Cheers,
>>
>> Taher Alkhateeb
>>
>> [1] https://issues.apache.org/jira/browse/OFBIZ-9322
>> [2] https://lists.apache.org/thread.html/a18b5d4086a07ff34a0cb95
>> 0999e8f8e869a6aa7efa2fc866a75a7b6@%3Cdev.ofbiz.apache.org%3E
>> [3] directory structure:
>> data/
>> ├── demo
>> │   ├── accounting-demo.xml
>> │   ├── common-demo.xml
>> │   ├── content-demo.xml
>> │   ├── humanres-demo.xml
>> │   ├── manufacturing-demo.xml
>> │   ├── marketing-demo.xml
>> │   ├── order-demo.xml
>> │   ├── party-demo.xml
>> │   ├── product-demo.xml
>> │   ├── shipment-demo.xml
>> │   └── workeffort-demo.xml
>> ├── seed
>> │   ├── accounting-seed.xml
>> │   ├── common-seed.xml
>> │   ├── content-seed.xml
>> │   ├── humanres-seed.xml
>> │   ├── manufacturing-seed.xml
>> │   ├── marketing-seed.xml
>> │   ├── order-seed.xml
>> │   ├── party-seed.xml
>> │   ├── product-seed.xml
>> │   ├── shipment-seed.xml
>> │   └── workeffort-seed.xml
>> └── seed-initial
>>      ├── accounting-seed-initial.xml
>>      ├── common-seed-initial.xml
>>      ├── content-seed-initial.xml
>>      ├── humanres-seed-initial.xml
>>      ├── manufacturing-seed-initial.xml
>>      ├── marketing-seed-initial.xml
>>      ├── order-seed-initial.xml
>>      ├── party-seed-initial.xml
>>      ├── product-seed-initial.xml
>>      ├── shipment-seed-initial.xml
>>      └── workeffort-seed-initial.xml
>>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [Proposal] moving all data in applications to the data-model component

Ashish Vijaywargiya-4
In reply to this post by taher
+1

--
Kind Regards
Ashish Vijaywargiya
HotWax Systems - est. 1997 <http://www.hotwaxsystems.com/>


On Fri, Jul 14, 2017 at 3:29 PM, Taher Alkhateeb <[hidden email]
> wrote:

>  Hello Everyone,
>
> Data loading in OFBiz is quite fragile. A small change could crash the
> system and finding out the reason can be difficult. I really felt the
> pain of this when trying to disentangle the framework from data
> dependencies on plugins [1].
>
> I've been thinking about a solution to reduce fragility, and here are
> my thoughts:
> - The less data files the better.
> - The less data the better.
> - The less distance (e.g. same directory) between data files the better.
> - No circular dependencies (file dependencies should be like a tree).
> - Fail fast [2].
>
> Therefore, I suggest the following action points:
> - Create a directory structure similar to [3] in the datamodel component.
> - For any data that two or more files depend on, immediately move it
> to the common-<reader>.xml file.
> - For any data that does not belong to any domain (e.g. marketing,
> content, etc ..) move it to the common-<reader>.xml.
> - Maintain a strict dependency hierarchy of demo -> seed -> seed-initial.
> - Gradually and carefully move data from all applications to the
> datamodel component as per above guidelines.
> - Cleanup and refactor the data as we move it including any tests that
> depend on it.
>
> What do you think of this proposal? Does it sound reasonable? Also
> given the size of what I'm proposing I think we need multiple people
> to work on this. So is anyone interested in taking part in this
> initiative?
>
> Cheers,
>
> Taher Alkhateeb
>
> [1] https://issues.apache.org/jira/browse/OFBIZ-9322
> [2] https://lists.apache.org/thread.html/a18b5d4086a07ff34a0cb950999e8f
> 8e869a6aa7efa2fc866a75a7b6@%3Cdev.ofbiz.apache.org%3E
> [3] directory structure:
> data/
> ├── demo
> │   ├── accounting-demo.xml
> │   ├── common-demo.xml
> │   ├── content-demo.xml
> │   ├── humanres-demo.xml
> │   ├── manufacturing-demo.xml
> │   ├── marketing-demo.xml
> │   ├── order-demo.xml
> │   ├── party-demo.xml
> │   ├── product-demo.xml
> │   ├── shipment-demo.xml
> │   └── workeffort-demo.xml
> ├── seed
> │   ├── accounting-seed.xml
> │   ├── common-seed.xml
> │   ├── content-seed.xml
> │   ├── humanres-seed.xml
> │   ├── manufacturing-seed.xml
> │   ├── marketing-seed.xml
> │   ├── order-seed.xml
> │   ├── party-seed.xml
> │   ├── product-seed.xml
> │   ├── shipment-seed.xml
> │   └── workeffort-seed.xml
> └── seed-initial
>     ├── accounting-seed-initial.xml
>     ├── common-seed-initial.xml
>     ├── content-seed-initial.xml
>     ├── humanres-seed-initial.xml
>     ├── manufacturing-seed-initial.xml
>     ├── marketing-seed-initial.xml
>     ├── order-seed-initial.xml
>     ├── party-seed-initial.xml
>     ├── product-seed-initial.xml
>     ├── shipment-seed-initial.xml
>     └── workeffort-seed-initial.xml
>
Reply | Threaded
Open this post in threaded view
|

Re: [Proposal] moving all data in applications to the data-model component

Nicolas Malin-2
In reply to this post by taher
+1


Le 14/07/2017 à 11:59, Taher Alkhateeb a écrit :

>   Hello Everyone,
>
> Data loading in OFBiz is quite fragile. A small change could crash the
> system and finding out the reason can be difficult. I really felt the
> pain of this when trying to disentangle the framework from data
> dependencies on plugins [1].
>
> I've been thinking about a solution to reduce fragility, and here are
> my thoughts:
> - The less data files the better.
> - The less data the better.
> - The less distance (e.g. same directory) between data files the better.
> - No circular dependencies (file dependencies should be like a tree).
> - Fail fast [2].
>
> Therefore, I suggest the following action points:
> - Create a directory structure similar to [3] in the datamodel component.
> - For any data that two or more files depend on, immediately move it
> to the common-<reader>.xml file.
> - For any data that does not belong to any domain (e.g. marketing,
> content, etc ..) move it to the common-<reader>.xml.
> - Maintain a strict dependency hierarchy of demo -> seed -> seed-initial.
> - Gradually and carefully move data from all applications to the
> datamodel component as per above guidelines.
> - Cleanup and refactor the data as we move it including any tests that
> depend on it.
>
> What do you think of this proposal? Does it sound reasonable? Also
> given the size of what I'm proposing I think we need multiple people
> to work on this. So is anyone interested in taking part in this
> initiative?
>
> Cheers,
>
> Taher Alkhateeb
>
> [1] https://issues.apache.org/jira/browse/OFBIZ-9322
> [2] https://lists.apache.org/thread.html/a18b5d4086a07ff34a0cb950999e8f8e869a6aa7efa2fc866a75a7b6@%3Cdev.ofbiz.apache.org%3E
> [3] directory structure:
> data/
> ├── demo
> │   ├── accounting-demo.xml
> │   ├── common-demo.xml
> │   ├── content-demo.xml
> │   ├── humanres-demo.xml
> │   ├── manufacturing-demo.xml
> │   ├── marketing-demo.xml
> │   ├── order-demo.xml
> │   ├── party-demo.xml
> │   ├── product-demo.xml
> │   ├── shipment-demo.xml
> │   └── workeffort-demo.xml
> ├── seed
> │   ├── accounting-seed.xml
> │   ├── common-seed.xml
> │   ├── content-seed.xml
> │   ├── humanres-seed.xml
> │   ├── manufacturing-seed.xml
> │   ├── marketing-seed.xml
> │   ├── order-seed.xml
> │   ├── party-seed.xml
> │   ├── product-seed.xml
> │   ├── shipment-seed.xml
> │   └── workeffort-seed.xml
> └── seed-initial
>      ├── accounting-seed-initial.xml
>      ├── common-seed-initial.xml
>      ├── content-seed-initial.xml
>      ├── humanres-seed-initial.xml
>      ├── manufacturing-seed-initial.xml
>      ├── marketing-seed-initial.xml
>      ├── order-seed-initial.xml
>      ├── party-seed-initial.xml
>      ├── product-seed-initial.xml
>      ├── shipment-seed-initial.xml
>      └── workeffort-seed-initial.xml
>

Reply | Threaded
Open this post in threaded view
|

Re: [Proposal] moving all data in applications to the data-model component

taher
Thank you all for your feedback and votes. I just counted 12 positive
votes to go ahead.

I have created a JIRA [1] for this task and I will work on it, but it
would be fantastic if someone is willing to take the lead on this so
that I can focus on refactoring the rest of the core framework. Also
any volunteers to commit or provide patches are more than welcomed.

[1] https://issues.apache.org/jira/browse/OFBIZ-9501

On Tue, Jul 18, 2017 at 6:57 PM, Nicolas Malin <[hidden email]> wrote:

> +1
>
>
>
> Le 14/07/2017 à 11:59, Taher Alkhateeb a écrit :
>>
>>   Hello Everyone,
>>
>> Data loading in OFBiz is quite fragile. A small change could crash the
>> system and finding out the reason can be difficult. I really felt the
>> pain of this when trying to disentangle the framework from data
>> dependencies on plugins [1].
>>
>> I've been thinking about a solution to reduce fragility, and here are
>> my thoughts:
>> - The less data files the better.
>> - The less data the better.
>> - The less distance (e.g. same directory) between data files the better.
>> - No circular dependencies (file dependencies should be like a tree).
>> - Fail fast [2].
>>
>> Therefore, I suggest the following action points:
>> - Create a directory structure similar to [3] in the datamodel component.
>> - For any data that two or more files depend on, immediately move it
>> to the common-<reader>.xml file.
>> - For any data that does not belong to any domain (e.g. marketing,
>> content, etc ..) move it to the common-<reader>.xml.
>> - Maintain a strict dependency hierarchy of demo -> seed -> seed-initial.
>> - Gradually and carefully move data from all applications to the
>> datamodel component as per above guidelines.
>> - Cleanup and refactor the data as we move it including any tests that
>> depend on it.
>>
>> What do you think of this proposal? Does it sound reasonable? Also
>> given the size of what I'm proposing I think we need multiple people
>> to work on this. So is anyone interested in taking part in this
>> initiative?
>>
>> Cheers,
>>
>> Taher Alkhateeb
>>
>> [1] https://issues.apache.org/jira/browse/OFBIZ-9322
>> [2]
>> https://lists.apache.org/thread.html/a18b5d4086a07ff34a0cb950999e8f8e869a6aa7efa2fc866a75a7b6@%3Cdev.ofbiz.apache.org%3E
>> [3] directory structure:
>> data/
>> ├── demo
>> │   ├── accounting-demo.xml
>> │   ├── common-demo.xml
>> │   ├── content-demo.xml
>> │   ├── humanres-demo.xml
>> │   ├── manufacturing-demo.xml
>> │   ├── marketing-demo.xml
>> │   ├── order-demo.xml
>> │   ├── party-demo.xml
>> │   ├── product-demo.xml
>> │   ├── shipment-demo.xml
>> │   └── workeffort-demo.xml
>> ├── seed
>> │   ├── accounting-seed.xml
>> │   ├── common-seed.xml
>> │   ├── content-seed.xml
>> │   ├── humanres-seed.xml
>> │   ├── manufacturing-seed.xml
>> │   ├── marketing-seed.xml
>> │   ├── order-seed.xml
>> │   ├── party-seed.xml
>> │   ├── product-seed.xml
>> │   ├── shipment-seed.xml
>> │   └── workeffort-seed.xml
>> └── seed-initial
>>      ├── accounting-seed-initial.xml
>>      ├── common-seed-initial.xml
>>      ├── content-seed-initial.xml
>>      ├── humanres-seed-initial.xml
>>      ├── manufacturing-seed-initial.xml
>>      ├── marketing-seed-initial.xml
>>      ├── order-seed-initial.xml
>>      ├── party-seed-initial.xml
>>      ├── product-seed-initial.xml
>>      ├── shipment-seed-initial.xml
>>      └── workeffort-seed-initial.xml
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: [Proposal] moving all data in applications to the data-model component

Swapnil Mane
Hello Taher,

Thanks for the initiative, please count me in. I will help in providing
patches for the same.


- Best Regards,
Swapnil M Mane
www.hotwaxsystems.com
www.hotwax.co

On Thu, Jul 20, 2017 at 12:15 PM, Taher Alkhateeb <
[hidden email]> wrote:

> Thank you all for your feedback and votes. I just counted 12 positive
> votes to go ahead.
>
> I have created a JIRA [1] for this task and I will work on it, but it
> would be fantastic if someone is willing to take the lead on this so
> that I can focus on refactoring the rest of the core framework. Also
> any volunteers to commit or provide patches are more than welcomed.
>
> [1] https://issues.apache.org/jira/browse/OFBIZ-9501
>
> On Tue, Jul 18, 2017 at 6:57 PM, Nicolas Malin <[hidden email]>
> wrote:
> > +1
> >
> >
> >
> > Le 14/07/2017 à 11:59, Taher Alkhateeb a écrit :
> >>
> >>   Hello Everyone,
> >>
> >> Data loading in OFBiz is quite fragile. A small change could crash the
> >> system and finding out the reason can be difficult. I really felt the
> >> pain of this when trying to disentangle the framework from data
> >> dependencies on plugins [1].
> >>
> >> I've been thinking about a solution to reduce fragility, and here are
> >> my thoughts:
> >> - The less data files the better.
> >> - The less data the better.
> >> - The less distance (e.g. same directory) between data files the better.
> >> - No circular dependencies (file dependencies should be like a tree).
> >> - Fail fast [2].
> >>
> >> Therefore, I suggest the following action points:
> >> - Create a directory structure similar to [3] in the datamodel
> component.
> >> - For any data that two or more files depend on, immediately move it
> >> to the common-<reader>.xml file.
> >> - For any data that does not belong to any domain (e.g. marketing,
> >> content, etc ..) move it to the common-<reader>.xml.
> >> - Maintain a strict dependency hierarchy of demo -> seed ->
> seed-initial.
> >> - Gradually and carefully move data from all applications to the
> >> datamodel component as per above guidelines.
> >> - Cleanup and refactor the data as we move it including any tests that
> >> depend on it.
> >>
> >> What do you think of this proposal? Does it sound reasonable? Also
> >> given the size of what I'm proposing I think we need multiple people
> >> to work on this. So is anyone interested in taking part in this
> >> initiative?
> >>
> >> Cheers,
> >>
> >> Taher Alkhateeb
> >>
> >> [1] https://issues.apache.org/jira/browse/OFBIZ-9322
> >> [2]
> >> https://lists.apache.org/thread.html/a18b5d4086a07ff34a0cb950999e8f
> 8e869a6aa7efa2fc866a75a7b6@%3Cdev.ofbiz.apache.org%3E
> >> [3] directory structure:
> >> data/
> >> ├── demo
> >> │   ├── accounting-demo.xml
> >> │   ├── common-demo.xml
> >> │   ├── content-demo.xml
> >> │   ├── humanres-demo.xml
> >> │   ├── manufacturing-demo.xml
> >> │   ├── marketing-demo.xml
> >> │   ├── order-demo.xml
> >> │   ├── party-demo.xml
> >> │   ├── product-demo.xml
> >> │   ├── shipment-demo.xml
> >> │   └── workeffort-demo.xml
> >> ├── seed
> >> │   ├── accounting-seed.xml
> >> │   ├── common-seed.xml
> >> │   ├── content-seed.xml
> >> │   ├── humanres-seed.xml
> >> │   ├── manufacturing-seed.xml
> >> │   ├── marketing-seed.xml
> >> │   ├── order-seed.xml
> >> │   ├── party-seed.xml
> >> │   ├── product-seed.xml
> >> │   ├── shipment-seed.xml
> >> │   └── workeffort-seed.xml
> >> └── seed-initial
> >>      ├── accounting-seed-initial.xml
> >>      ├── common-seed-initial.xml
> >>      ├── content-seed-initial.xml
> >>      ├── humanres-seed-initial.xml
> >>      ├── manufacturing-seed-initial.xml
> >>      ├── marketing-seed-initial.xml
> >>      ├── order-seed-initial.xml
> >>      ├── party-seed-initial.xml
> >>      ├── product-seed-initial.xml
> >>      ├── shipment-seed-initial.xml
> >>      └── workeffort-seed-initial.xml
> >>
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: [Proposal] moving all data in applications to the data-model component

taher
That's super Swapnil, thank you :)

On Thu, Jul 20, 2017 at 10:47 AM, Swapnil Mane
<[hidden email]> wrote:

> Hello Taher,
>
> Thanks for the initiative, please count me in. I will help in providing
> patches for the same.
>
>
> - Best Regards,
> Swapnil M Mane
> www.hotwaxsystems.com
> www.hotwax.co
>
> On Thu, Jul 20, 2017 at 12:15 PM, Taher Alkhateeb <
> [hidden email]> wrote:
>
>> Thank you all for your feedback and votes. I just counted 12 positive
>> votes to go ahead.
>>
>> I have created a JIRA [1] for this task and I will work on it, but it
>> would be fantastic if someone is willing to take the lead on this so
>> that I can focus on refactoring the rest of the core framework. Also
>> any volunteers to commit or provide patches are more than welcomed.
>>
>> [1] https://issues.apache.org/jira/browse/OFBIZ-9501
>>
>> On Tue, Jul 18, 2017 at 6:57 PM, Nicolas Malin <[hidden email]>
>> wrote:
>> > +1
>> >
>> >
>> >
>> > Le 14/07/2017 à 11:59, Taher Alkhateeb a écrit :
>> >>
>> >>   Hello Everyone,
>> >>
>> >> Data loading in OFBiz is quite fragile. A small change could crash the
>> >> system and finding out the reason can be difficult. I really felt the
>> >> pain of this when trying to disentangle the framework from data
>> >> dependencies on plugins [1].
>> >>
>> >> I've been thinking about a solution to reduce fragility, and here are
>> >> my thoughts:
>> >> - The less data files the better.
>> >> - The less data the better.
>> >> - The less distance (e.g. same directory) between data files the better.
>> >> - No circular dependencies (file dependencies should be like a tree).
>> >> - Fail fast [2].
>> >>
>> >> Therefore, I suggest the following action points:
>> >> - Create a directory structure similar to [3] in the datamodel
>> component.
>> >> - For any data that two or more files depend on, immediately move it
>> >> to the common-<reader>.xml file.
>> >> - For any data that does not belong to any domain (e.g. marketing,
>> >> content, etc ..) move it to the common-<reader>.xml.
>> >> - Maintain a strict dependency hierarchy of demo -> seed ->
>> seed-initial.
>> >> - Gradually and carefully move data from all applications to the
>> >> datamodel component as per above guidelines.
>> >> - Cleanup and refactor the data as we move it including any tests that
>> >> depend on it.
>> >>
>> >> What do you think of this proposal? Does it sound reasonable? Also
>> >> given the size of what I'm proposing I think we need multiple people
>> >> to work on this. So is anyone interested in taking part in this
>> >> initiative?
>> >>
>> >> Cheers,
>> >>
>> >> Taher Alkhateeb
>> >>
>> >> [1] https://issues.apache.org/jira/browse/OFBIZ-9322
>> >> [2]
>> >> https://lists.apache.org/thread.html/a18b5d4086a07ff34a0cb950999e8f
>> 8e869a6aa7efa2fc866a75a7b6@%3Cdev.ofbiz.apache.org%3E
>> >> [3] directory structure:
>> >> data/
>> >> ├── demo
>> >> │   ├── accounting-demo.xml
>> >> │   ├── common-demo.xml
>> >> │   ├── content-demo.xml
>> >> │   ├── humanres-demo.xml
>> >> │   ├── manufacturing-demo.xml
>> >> │   ├── marketing-demo.xml
>> >> │   ├── order-demo.xml
>> >> │   ├── party-demo.xml
>> >> │   ├── product-demo.xml
>> >> │   ├── shipment-demo.xml
>> >> │   └── workeffort-demo.xml
>> >> ├── seed
>> >> │   ├── accounting-seed.xml
>> >> │   ├── common-seed.xml
>> >> │   ├── content-seed.xml
>> >> │   ├── humanres-seed.xml
>> >> │   ├── manufacturing-seed.xml
>> >> │   ├── marketing-seed.xml
>> >> │   ├── order-seed.xml
>> >> │   ├── party-seed.xml
>> >> │   ├── product-seed.xml
>> >> │   ├── shipment-seed.xml
>> >> │   └── workeffort-seed.xml
>> >> └── seed-initial
>> >>      ├── accounting-seed-initial.xml
>> >>      ├── common-seed-initial.xml
>> >>      ├── content-seed-initial.xml
>> >>      ├── humanres-seed-initial.xml
>> >>      ├── manufacturing-seed-initial.xml
>> >>      ├── marketing-seed-initial.xml
>> >>      ├── order-seed-initial.xml
>> >>      ├── party-seed-initial.xml
>> >>      ├── product-seed-initial.xml
>> >>      ├── shipment-seed-initial.xml
>> >>      └── workeffort-seed-initial.xml
>> >>
>> >
>>
Reply | Threaded
Open this post in threaded view
|

Re: [Proposal] moving all data in applications to the data-model component

Devanshu Vyas-2
In reply to this post by taher
Hello Taher,

I would like to work on to provide patches for this effort.

Thanks & Regards,
Devanshu Vyas.

On Thu, Jul 20, 2017 at 12:15 PM, Taher Alkhateeb <
[hidden email]> wrote:

> Thank you all for your feedback and votes. I just counted 12 positive
> votes to go ahead.
>
> I have created a JIRA [1] for this task and I will work on it, but it
> would be fantastic if someone is willing to take the lead on this so
> that I can focus on refactoring the rest of the core framework. Also
> any volunteers to commit or provide patches are more than welcomed.
>
> [1] https://issues.apache.org/jira/browse/OFBIZ-9501
>
> On Tue, Jul 18, 2017 at 6:57 PM, Nicolas Malin <[hidden email]>
> wrote:
> > +1
> >
> >
> >
> > Le 14/07/2017 à 11:59, Taher Alkhateeb a écrit :
> >>
> >>   Hello Everyone,
> >>
> >> Data loading in OFBiz is quite fragile. A small change could crash the
> >> system and finding out the reason can be difficult. I really felt the
> >> pain of this when trying to disentangle the framework from data
> >> dependencies on plugins [1].
> >>
> >> I've been thinking about a solution to reduce fragility, and here are
> >> my thoughts:
> >> - The less data files the better.
> >> - The less data the better.
> >> - The less distance (e.g. same directory) between data files the better.
> >> - No circular dependencies (file dependencies should be like a tree).
> >> - Fail fast [2].
> >>
> >> Therefore, I suggest the following action points:
> >> - Create a directory structure similar to [3] in the datamodel
> component.
> >> - For any data that two or more files depend on, immediately move it
> >> to the common-<reader>.xml file.
> >> - For any data that does not belong to any domain (e.g. marketing,
> >> content, etc ..) move it to the common-<reader>.xml.
> >> - Maintain a strict dependency hierarchy of demo -> seed ->
> seed-initial.
> >> - Gradually and carefully move data from all applications to the
> >> datamodel component as per above guidelines.
> >> - Cleanup and refactor the data as we move it including any tests that
> >> depend on it.
> >>
> >> What do you think of this proposal? Does it sound reasonable? Also
> >> given the size of what I'm proposing I think we need multiple people
> >> to work on this. So is anyone interested in taking part in this
> >> initiative?
> >>
> >> Cheers,
> >>
> >> Taher Alkhateeb
> >>
> >> [1] https://issues.apache.org/jira/browse/OFBIZ-9322
> >> [2]
> >> https://lists.apache.org/thread.html/a18b5d4086a07ff34a0cb950999e8f
> 8e869a6aa7efa2fc866a75a7b6@%3Cdev.ofbiz.apache.org%3E
> >> [3] directory structure:
> >> data/
> >> ├── demo
> >> │   ├── accounting-demo.xml
> >> │   ├── common-demo.xml
> >> │   ├── content-demo.xml
> >> │   ├── humanres-demo.xml
> >> │   ├── manufacturing-demo.xml
> >> │   ├── marketing-demo.xml
> >> │   ├── order-demo.xml
> >> │   ├── party-demo.xml
> >> │   ├── product-demo.xml
> >> │   ├── shipment-demo.xml
> >> │   └── workeffort-demo.xml
> >> ├── seed
> >> │   ├── accounting-seed.xml
> >> │   ├── common-seed.xml
> >> │   ├── content-seed.xml
> >> │   ├── humanres-seed.xml
> >> │   ├── manufacturing-seed.xml
> >> │   ├── marketing-seed.xml
> >> │   ├── order-seed.xml
> >> │   ├── party-seed.xml
> >> │   ├── product-seed.xml
> >> │   ├── shipment-seed.xml
> >> │   └── workeffort-seed.xml
> >> └── seed-initial
> >>      ├── accounting-seed-initial.xml
> >>      ├── common-seed-initial.xml
> >>      ├── content-seed-initial.xml
> >>      ├── humanres-seed-initial.xml
> >>      ├── manufacturing-seed-initial.xml
> >>      ├── marketing-seed-initial.xml
> >>      ├── order-seed-initial.xml
> >>      ├── party-seed-initial.xml
> >>      ├── product-seed-initial.xml
> >>      ├── shipment-seed-initial.xml
> >>      └── workeffort-seed-initial.xml
> >>
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: [Proposal] moving all data in applications to the data-model component

taher
Wonderful :) looking forward to getting this thing done

On Thu, Jul 20, 2017 at 11:31 AM, Devanshu Vyas
<[hidden email]> wrote:

> Hello Taher,
>
> I would like to work on to provide patches for this effort.
>
> Thanks & Regards,
> Devanshu Vyas.
>
> On Thu, Jul 20, 2017 at 12:15 PM, Taher Alkhateeb <
> [hidden email]> wrote:
>
>> Thank you all for your feedback and votes. I just counted 12 positive
>> votes to go ahead.
>>
>> I have created a JIRA [1] for this task and I will work on it, but it
>> would be fantastic if someone is willing to take the lead on this so
>> that I can focus on refactoring the rest of the core framework. Also
>> any volunteers to commit or provide patches are more than welcomed.
>>
>> [1] https://issues.apache.org/jira/browse/OFBIZ-9501
>>
>> On Tue, Jul 18, 2017 at 6:57 PM, Nicolas Malin <[hidden email]>
>> wrote:
>> > +1
>> >
>> >
>> >
>> > Le 14/07/2017 à 11:59, Taher Alkhateeb a écrit :
>> >>
>> >>   Hello Everyone,
>> >>
>> >> Data loading in OFBiz is quite fragile. A small change could crash the
>> >> system and finding out the reason can be difficult. I really felt the
>> >> pain of this when trying to disentangle the framework from data
>> >> dependencies on plugins [1].
>> >>
>> >> I've been thinking about a solution to reduce fragility, and here are
>> >> my thoughts:
>> >> - The less data files the better.
>> >> - The less data the better.
>> >> - The less distance (e.g. same directory) between data files the better.
>> >> - No circular dependencies (file dependencies should be like a tree).
>> >> - Fail fast [2].
>> >>
>> >> Therefore, I suggest the following action points:
>> >> - Create a directory structure similar to [3] in the datamodel
>> component.
>> >> - For any data that two or more files depend on, immediately move it
>> >> to the common-<reader>.xml file.
>> >> - For any data that does not belong to any domain (e.g. marketing,
>> >> content, etc ..) move it to the common-<reader>.xml.
>> >> - Maintain a strict dependency hierarchy of demo -> seed ->
>> seed-initial.
>> >> - Gradually and carefully move data from all applications to the
>> >> datamodel component as per above guidelines.
>> >> - Cleanup and refactor the data as we move it including any tests that
>> >> depend on it.
>> >>
>> >> What do you think of this proposal? Does it sound reasonable? Also
>> >> given the size of what I'm proposing I think we need multiple people
>> >> to work on this. So is anyone interested in taking part in this
>> >> initiative?
>> >>
>> >> Cheers,
>> >>
>> >> Taher Alkhateeb
>> >>
>> >> [1] https://issues.apache.org/jira/browse/OFBIZ-9322
>> >> [2]
>> >> https://lists.apache.org/thread.html/a18b5d4086a07ff34a0cb950999e8f
>> 8e869a6aa7efa2fc866a75a7b6@%3Cdev.ofbiz.apache.org%3E
>> >> [3] directory structure:
>> >> data/
>> >> ├── demo
>> >> │   ├── accounting-demo.xml
>> >> │   ├── common-demo.xml
>> >> │   ├── content-demo.xml
>> >> │   ├── humanres-demo.xml
>> >> │   ├── manufacturing-demo.xml
>> >> │   ├── marketing-demo.xml
>> >> │   ├── order-demo.xml
>> >> │   ├── party-demo.xml
>> >> │   ├── product-demo.xml
>> >> │   ├── shipment-demo.xml
>> >> │   └── workeffort-demo.xml
>> >> ├── seed
>> >> │   ├── accounting-seed.xml
>> >> │   ├── common-seed.xml
>> >> │   ├── content-seed.xml
>> >> │   ├── humanres-seed.xml
>> >> │   ├── manufacturing-seed.xml
>> >> │   ├── marketing-seed.xml
>> >> │   ├── order-seed.xml
>> >> │   ├── party-seed.xml
>> >> │   ├── product-seed.xml
>> >> │   ├── shipment-seed.xml
>> >> │   └── workeffort-seed.xml
>> >> └── seed-initial
>> >>      ├── accounting-seed-initial.xml
>> >>      ├── common-seed-initial.xml
>> >>      ├── content-seed-initial.xml
>> >>      ├── humanres-seed-initial.xml
>> >>      ├── manufacturing-seed-initial.xml
>> >>      ├── marketing-seed-initial.xml
>> >>      ├── order-seed-initial.xml
>> >>      ├── party-seed-initial.xml
>> >>      ├── product-seed-initial.xml
>> >>      ├── shipment-seed-initial.xml
>> >>      └── workeffort-seed-initial.xml
>> >>
>> >
>>