Why A Framework Rewrite Is Necessary
Locked
123
123
Re: Why A Framework Rewrite Is Necessary
 
|
I don't think this community is large enough to attempt to rewrite all of
our applications to suit Moqui while at the same time still supporting them under the existing framework. That may well be the case for a homegrown framework rewrite as well depending on how the applications would be affected. This is the main reason why I tend to prefer incremental improvements to the existing framework. But if people within the community wanted to try for either of the "new framework" options, I certainly don't have a problem with it and am happy to see some infrastructure donated to the cause. There's simply no need to form a consensus on any major decision in the early stages, proponents just need to be willing to accept that their endeavors might not become part of the project. David certainly didn't wait until he had a community backing him to go ahead and start work on Moqui. The best way to get something done is to get started on it, if no one follows then it's either a bad idea or you're doing a bad job of selling it. By the way, I'm surprised I haven't seen any efforts within the Moqui community to rewrite any of the OFBiz apps. I don't have any ideas as to why that hasn't happened, but I'm curious about it. Surely it would be easier for the Moqui community who already know the framework and are probably also familiar with OFBiz to try this than us. Regards Scott On 16 October 2015 at 12:20, Hans Bakker <[hidden email]> wrote: > Well said: +1 > > > On 15/10/15 22:13, Al Byers wrote: > >> Fair enough, Adrian, but is this a widely held position on OFBiz that as a >> group you could produce a product that is so much better than Moqui that >> it >> would be worth the time and risk (the risk of burning cycles that you will >> never get back only to have the venture never succeed.) Have all the >> people >> who would like to do a rewrite done the same gap analysis on Moqui vs what >> you would like to see OFBiz be? Does everyone understand and agree that >> the >> deficiencies that you see in Moqui are important? My concern is that the >> many people who understandably want a rewrite of OFBiz will be swayed by >> your arguments without there being due diligence being done. I think that >> there are a lot of veterans of OFBiz who have confidence in David's >> development skills, but who are not pressing for a move to Moqui or for a >> rewrite because they have so much invested in the current state of OFBiz >> and recognized what a long shot a rewrite would be and how much work it >> would require of them to move over. But anybody who is open to a rewrite >> would be foolish not to consider the work done by the principal architect >> of OFBiz; work that was meant to address deficiencies in OFBiz. >> >> On Thu, Oct 15, 2015 at 7:58 AM, Adrian Crum < >> [hidden email]> wrote: >> >> Keep in mind that much of David's code in OFBiz has been rewritten. So >>> yes, we CAN do a better job than him. Also keep in mind that Moqui >>> duplicates some of the problems I listed - so by using Moqui, we keep the >>> same problems instead of fixing them. On the other hand, it is David's >>> responsibility to fix them, not ours. >>> >>> If we created a sub-project, then we would have the opportunity to review >>> committer permissions and perhaps restrict access to the new code. >>> >>> Adrian Crum >>> Sandglass Software >>> www.sandglass-software.com >>> >>> On 10/15/2015 6:34 AM, Al Byers wrote: >>> >>> I was waiting for someone to bring this up. David Jones created Moqui >>>> with >>>> the same end in mind that the rewrite is meant to accomplish. Do you >>>> think >>>> that you will do a better job than him? Even if you could, would it be >>>> so >>>> much better that it warrants the effort that it would take? >>>> >>>> Is this a political thing? I don't know that David would give up control >>>> of >>>> Moqui Core and I, for one, would not want him to. So many of OFBiz's >>>> problems are the result of ineptitude on the part of committers who did >>>> not >>>> know what they were doing (like me.) But I don't know that the Core will >>>> change all that much going forward and there should be places in the >>>> Mantle >>>> to contribute and, most certainly, in the Crust. >>>> >>>> I know that there were some valid questions about licensing and project >>>> management, but I would think that they could be worked out. >>>> >>>> On Thu, Oct 15, 2015 at 4:46 AM, Hans Bakker < >>>> [hidden email]> >>>> wrote: >>>> >>>> Why not skip this step, using moqui which is already up and running and >>>> >>>>> then as Adrian states as a next step: applications could be pulled down >>>>> and >>>>> adapted to it. >>>>> >>>>> Hans >>>>> >>>>> >>>>> >>>>> On 15/10/15 16:51, Jacques Le Roux wrote: >>>>> >>>>> I'm in the same mood than Paul and Scott. So a sub-project could indeed >>>>> >>>>>> be a solution. >>>>>> >>>>>> Jacques >>>>>> >>>>>> Le 15/10/2015 03:11, Adrian Crum a écrit : >>>>>> >>>>>> I agree that a sub-project would be nice to have. Once the new >>>>>> framework >>>>>> >>>>>>> is up and running, applications could be pulled down and adapted to >>>>>>> it. >>>>>>> >>>>>>> Adrian Crum >>>>>>> Sandglass Software >>>>>>> www.sandglass-software.com >>>>>>> >>>>>>> On 10/14/2015 5:53 PM, Ron Wheeler wrote: >>>>>>> >>>>>>> This seems to be very good advice. >>>>>>> >>>>>>>> A completely separate sub-project under OFBiz with its own mailing >>>>>>>> lists >>>>>>>> would keep the people together yet all allow the new framework the >>>>>>>> flexibility to move forward. >>>>>>>> >>>>>>>> Ron >>>>>>>> On 14/10/2015 8:27 PM, Scott Gray wrote: >>>>>>>> >>>>>>>> My advice is as follows: >>>>>>>> >>>>>>>>> 1. If people are interested, then get them together and start >>>>>>>>> working >>>>>>>>> on it. >>>>>>>>> 2. Find somewhere to do the work. I don't think a branch is >>>>>>>>> appropriate >>>>>>>>> because it's completely new development rather than a >>>>>>>>> refactoring. I >>>>>>>>> don't >>>>>>>>> have any objections to it being done under the ASF OFBiz umbrella >>>>>>>>> (although >>>>>>>>> I don't really see the need either). >>>>>>>>> 3. Set up a separate mailing list for discussion. Generally I'd >>>>>>>>> try >>>>>>>>> and >>>>>>>>> keep quiet about it in the early stages on the dev/user lists or >>>>>>>>> other >>>>>>>>> marketing channels because it could potentially harm adoption of >>>>>>>>> our >>>>>>>>> existing framework (think AngularJS 2.0). >>>>>>>>> >>>>>>>>> There really isn't any need to get early stage sign-off from the >>>>>>>>> PMC >>>>>>>>> or >>>>>>>>> anyone else in the community. You only need enough PMC approval to >>>>>>>>> get the >>>>>>>>> required infrastructure sorted, which I don't think would be an >>>>>>>>> issue. >>>>>>>>> >From there, it's really up to the community to decide whether or >>>>>>>>> not >>>>>>>>> the >>>>>>>>> thing will fly. >>>>>>>>> >>>>>>>>> Regards >>>>>>>>> Scott >>>>>>>>> >>>>>>>>> >>>>>>>>> On 15 October 2015 at 08:21, Adrian Crum >>>>>>>>> <[hidden email] >>>>>>>>> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> I understand that Sharan brought up the framework rewrite subject >>>>>>>>>> at >>>>>>>>>> ApacheCon, and some attendees felt that the framework is fine and >>>>>>>>>> no >>>>>>>>>> action >>>>>>>>>> needs to be taken. >>>>>>>>>> >>>>>>>>>> In this message, I will try to give a detailed explanation of why >>>>>>>>>> a >>>>>>>>>> framework rewrite is necessary. I don't plan to take any further >>>>>>>>>> action on >>>>>>>>>> this subject, because I've brought it up before without success, >>>>>>>>>> and >>>>>>>>>> I'm >>>>>>>>>> tired of discussing it. It is my hope that the light bulb will >>>>>>>>>> click >>>>>>>>>> on in >>>>>>>>>> someone's head and they will take action. >>>>>>>>>> >>>>>>>>>> My Background >>>>>>>>>> ------------- >>>>>>>>>> >>>>>>>>>> I became a member of the OFBiz community in 2004. I immediately >>>>>>>>>> started >>>>>>>>>> making contributions to the project by supplying patches to the >>>>>>>>>> issue >>>>>>>>>> tracker. In 2007, I became a committer. Most of my initial work >>>>>>>>>> was >>>>>>>>>> on the >>>>>>>>>> UI and some work in the applications (mainly Asset Maintenance and >>>>>>>>>> Work >>>>>>>>>> Effort). I stayed away from touching the framework code because it >>>>>>>>>> was >>>>>>>>>> deep, dark, and scary. >>>>>>>>>> >>>>>>>>>> Eventually, I started to understand how the framework code works >>>>>>>>>> and I >>>>>>>>>> made some minor modifications. As my understanding grew, I >>>>>>>>>> progressed >>>>>>>>>> to >>>>>>>>>> rewriting large swaths of framework code - making it thread-safe, >>>>>>>>>> fault >>>>>>>>>> tolerant, efficient, and easier to use. >>>>>>>>>> >>>>>>>>>> I will list some of my contributions here, so everyone can have a >>>>>>>>>> clear >>>>>>>>>> understanding of my experience with the framework code: >>>>>>>>>> >>>>>>>>>> New Features >>>>>>>>>> >>>>>>>>>> User Preferences >>>>>>>>>> >>>>>>>>>> Visual Themes >>>>>>>>>> >>>>>>>>>> Custom UI Label XML File Format >>>>>>>>>> >>>>>>>>>> Temporal Expressions >>>>>>>>>> >>>>>>>>>> Data Type Conversion Framework >>>>>>>>>> >>>>>>>>>> Screen Widget Boundary Comments >>>>>>>>>> >>>>>>>>>> Metrics >>>>>>>>>> >>>>>>>>>> Integrations >>>>>>>>>> >>>>>>>>>> UEL >>>>>>>>>> >>>>>>>>>> iCalendar >>>>>>>>>> >>>>>>>>>> JSR 223 >>>>>>>>>> >>>>>>>>>> WebDAV >>>>>>>>>> >>>>>>>>>> LDAP >>>>>>>>>> >>>>>>>>>> Refactorings/Improvements >>>>>>>>>> >>>>>>>>>> FlexibleStringExpander >>>>>>>>>> >>>>>>>>>> FlexibleMapExpander >>>>>>>>>> >>>>>>>>>> FOP Integration >>>>>>>>>> >>>>>>>>>> FreeMarkerWorker >>>>>>>>>> >>>>>>>>>> Date-Time Handling >>>>>>>>>> >>>>>>>>>> Mini-language >>>>>>>>>> >>>>>>>>>> Job Scheduler >>>>>>>>>> >>>>>>>>>> In addition, I have performed innumerable framework bug fixes. >>>>>>>>>> >>>>>>>>>> So, the contents of this message come from years of experience >>>>>>>>>> mucking >>>>>>>>>> about in the framework code. >>>>>>>>>> >>>>>>>>>> Okay, let's get started... >>>>>>>>>> >>>>>>>>>> Initial Problem Statement >>>>>>>>>> ------------------------- >>>>>>>>>> >>>>>>>>>> In 2009, David Jones started a framework rewrite in a branch: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> https://svn.apache.org/repos/asf/ofbiz/branches/executioncontext20090716 >>>>>>>>>> >>>>>>>>>> At the time, there was some agreement that a rewrite was >>>>>>>>>> necessary, >>>>>>>>>> but >>>>>>>>>> there was disagreement as to how the rewrite should be >>>>>>>>>> incorporated >>>>>>>>>> into >>>>>>>>>> the project: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> https://mail-archives.apache.org/mod_mbox/ofbiz-dev/200908.mbox/%3C455601.62605.qm@...%3E >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> There were concerns that a rewrite would break backward >>>>>>>>>> compatibility. >>>>>>>>>> Work on the rewrite branch stopped. Eventually, Jacopo suggested >>>>>>>>>> the >>>>>>>>>> community be more accepting of backward-incompatible changes: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> https://mail-archives.apache.org/mod_mbox/ofbiz-dev/201004.mbox/%3cD24F129D-4F9F-444E-84AF-ACA46F49990C@...%3e >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Despite an effort to convince David to proceed with the framework >>>>>>>>>> rewrite, >>>>>>>>>> he ended up doing it in a separate project: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> http://mail-archives.apache.org/mod_mbox/ofbiz-dev/201104.mbox/%3C07565C88-4023-4D24-93A3-A4906E86F939@...%3E >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> This page describes differences between OFBiz and Moqui, and >>>>>>>>>> within >>>>>>>>>> it you >>>>>>>>>> can extract information on the problems David was trying to solve: >>>>>>>>>> >>>>>>>>>> http://sourceforge.net/p/moqui/discussion/1086127/thread/4c52f240/ >>>>>>>>>> >>>>>>>>>> There was an email he sent out on the OFBiz dev list where he >>>>>>>>>> listed >>>>>>>>>> the >>>>>>>>>> problems he saw in the framework, but I can't find it. The rest of >>>>>>>>>> this >>>>>>>>>> message will include the issues he mentioned (the ones I >>>>>>>>>> remember). >>>>>>>>>> I >>>>>>>>>> was >>>>>>>>>> in agreement with him at the time, and I still agree that a >>>>>>>>>> framework >>>>>>>>>> rewrite is necessary. >>>>>>>>>> >>>>>>>>>> The Problems >>>>>>>>>> ------------ >>>>>>>>>> >>>>>>>>>> Code is scattered everywhere - due to an initial effort to make >>>>>>>>>> the >>>>>>>>>> framework modular. This causes serious problems. The mere fact >>>>>>>>>> that >>>>>>>>>> components like entityext and securityext EXIST makes it clear >>>>>>>>>> that >>>>>>>>>> there >>>>>>>>>> are problems - those components should not be there. Also, we run >>>>>>>>>> into the >>>>>>>>>> recurring problem of circular dependencies (component A will not >>>>>>>>>> build >>>>>>>>>> unless component B is built, and component B will not build unless >>>>>>>>>> component A is built). >>>>>>>>>> >>>>>>>>>> Bad separation of concerns. There are far too many examples of >>>>>>>>>> classes >>>>>>>>>> that try to be everything to everyone. This makes debugging >>>>>>>>>> difficult, and >>>>>>>>>> it makes maintenance/improvements a nightmare. [Using an analogy, >>>>>>>>>> consider >>>>>>>>>> an automobile design where a spark plug is not separate from a >>>>>>>>>> transmission. Instead, the automobile uses a >>>>>>>>>> spark-plug-transmission. >>>>>>>>>> So >>>>>>>>>> when the engine is running rough because the spark plug is bad, >>>>>>>>>> you >>>>>>>>>> have to >>>>>>>>>> replace the spark plug AND the transmission.] A good framework >>>>>>>>>> example can >>>>>>>>>> be found in my rewrite of the mini-language code. Originally, the >>>>>>>>>> models >>>>>>>>>> AND the script execution context both contained script behaviors - >>>>>>>>>> making >>>>>>>>>> debugging/improvements difficult. I changed it so only the models >>>>>>>>>> contain >>>>>>>>>> script behavior and the script execution context contains only the >>>>>>>>>> script >>>>>>>>>> execution state. >>>>>>>>>> >>>>>>>>>> Lack of good OO design. There are many places where a bit of >>>>>>>>>> framework >>>>>>>>>> functionality is contained in a single method that is hundreds or >>>>>>>>>> thousands >>>>>>>>>> of lines long. There is a term for that: Brittle Code. Code isn't >>>>>>>>>> reused. >>>>>>>>>> Instead, it is copy-and-pasted all over - so when a problem is >>>>>>>>>> found >>>>>>>>>> in the >>>>>>>>>> C&P code, it has to be fixed in many places instead of one. >>>>>>>>>> >>>>>>>>>> Fail-slow design. There are a lot of places in low-level code >>>>>>>>>> where >>>>>>>>>> an >>>>>>>>>> error condition is encountered, but instead of throwing an >>>>>>>>>> exception, >>>>>>>>>> the >>>>>>>>>> error is ignored and maybe it is logged, or the code tries to >>>>>>>>>> "guess" >>>>>>>>>> at a >>>>>>>>>> solution and then provide an arbitrary default behavior. I've seen >>>>>>>>>> many >>>>>>>>>> developers struggle with debugging a problem because they didn't >>>>>>>>>> look >>>>>>>>>> at >>>>>>>>>> the logs, or because the error was not logged and there is no way >>>>>>>>>> of >>>>>>>>>> knowing what caused it. They end up spending hours single-stepping >>>>>>>>>> through >>>>>>>>>> code until it reaches the error. >>>>>>>>>> >>>>>>>>>> Out-of-date code. A good example is the use of Javolution. That >>>>>>>>>> library >>>>>>>>>> was beneficial in the Java 1.4 days, but it is not necessary today >>>>>>>>>> because >>>>>>>>>> of improved garbage collection. Another good example is DCL code. >>>>>>>>>> DCL >>>>>>>>>> was >>>>>>>>>> used extensively in OFBiz, but it is clearly documented to be an >>>>>>>>>> unreliable >>>>>>>>>> design (I can get it to fail 90% of the time). Some DCL code has >>>>>>>>>> been >>>>>>>>>> replaced, but a lot of it is still there. >>>>>>>>>> >>>>>>>>>> Portions of the API are overly complicated. Some methods require a >>>>>>>>>> collection of user-specified artifacts/arguments, which makes >>>>>>>>>> client >>>>>>>>>> code >>>>>>>>>> complicated and verbose. (David solved that problem with his >>>>>>>>>> Execution >>>>>>>>>> Context.) Portions of the API are cluttered with unnecessary >>>>>>>>>> "convenience >>>>>>>>>> methods" - making the API harder to learn and memorize. In some >>>>>>>>>> places, a >>>>>>>>>> domain-specific API is spread across instance methods and static >>>>>>>>>> methods >>>>>>>>>> and across different classes - making the API hard to understand >>>>>>>>>> and >>>>>>>>>> use. >>>>>>>>>> Yes, there can be good designs that require something like that, >>>>>>>>>> but >>>>>>>>>> in the >>>>>>>>>> OFBiz framework, it exists because of a bad design, not a good >>>>>>>>>> one. >>>>>>>>>> >>>>>>>>>> Use of thread-local variables. This makes multi-threaded design >>>>>>>>>> impossible. The J2EE specification and the Servlet API require one >>>>>>>>>> thread >>>>>>>>>> per request (and most J2EE libraries depend on that behavior), so >>>>>>>>>> the >>>>>>>>>> current design makes sense from a J2EE perspective, but what if I >>>>>>>>>> don't >>>>>>>>>> want to run the framework in a J2EE container? Which leads to the >>>>>>>>>> next >>>>>>>>>> problem... >>>>>>>>>> >>>>>>>>>> Dependence on J2EE designs/APIs/libraries. There are developers in >>>>>>>>>> the >>>>>>>>>> Java community (myself included) who are beginning to question if >>>>>>>>>> J2EE is >>>>>>>>>> really necessary to run web applications. The folks at Atomikos >>>>>>>>>> are >>>>>>>>>> a >>>>>>>>>> good >>>>>>>>>> example. OFBiz does not use EJBs, so tying the framework to J2EE >>>>>>>>>> does >>>>>>>>>> not >>>>>>>>>> make sense. It would be better if the framework was designed to >>>>>>>>>> run >>>>>>>>>> outside >>>>>>>>>> a J2EE container, and then have container integration as an >>>>>>>>>> option. >>>>>>>>>> >>>>>>>>>> Configuration files are scattered everywhere. Anyone who has >>>>>>>>>> deployed >>>>>>>>>> OFBiz in a production environment will agree this is a problem. >>>>>>>>>> Try >>>>>>>>>> changing the HTTP/HTTPS and port settings - it is a nightmare. >>>>>>>>>> Some >>>>>>>>>> configuration settings are in nonsensical places. >>>>>>>>>> >>>>>>>>>> An abysmal lack of unit testing. I don't have an exact figure for >>>>>>>>>> code >>>>>>>>>> coverage, but my gut feeling is coverage is less than 10%. >>>>>>>>>> Basically, >>>>>>>>>> we >>>>>>>>>> all have our fingers crossed - hoping that the framework code >>>>>>>>>> works >>>>>>>>>> as >>>>>>>>>> expected. This was made painfully obvious a while back when I was >>>>>>>>>> looking >>>>>>>>>> at some entity caching code and thought to myself "this code can't >>>>>>>>>> work." >>>>>>>>>> So I wrote some entity cache unit tests and confirmed that the >>>>>>>>>> entity >>>>>>>>>> cache >>>>>>>>>> had serious problems. Think about that - years passed with no >>>>>>>>>> entity >>>>>>>>>> cache >>>>>>>>>> unit tests and consequently we had no idea it wasn't working. >>>>>>>>>> >>>>>>>>>> Fix Versus Rewrite >>>>>>>>>> ------------------ >>>>>>>>>> >>>>>>>>>> Jira issues could be created for these problems and teams of >>>>>>>>>> developers >>>>>>>>>> could work to fix them. >>>>>>>>>> >>>>>>>>>> Or, we could create a branch and start over from scratch. This >>>>>>>>>> time >>>>>>>>>> around, there should be less push-back from people concerned about >>>>>>>>>> backwards compatibility. A rewrite offers the advantage of >>>>>>>>>> reconsidering >>>>>>>>>> everything - like API design, general problem solving, and new >>>>>>>>>> features. >>>>>>>>>> >>>>>>>>>> I created a Wiki page for a framework design: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> https://cwiki.apache.org/confluence/display/OFBADMIN/Another+Framework+Vision >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> but there hasn't been much interest in it. If the community >>>>>>>>>> decides >>>>>>>>>> to go >>>>>>>>>> ahead with a rewrite, then please feel free to use the Wiki pages >>>>>>>>>> as a >>>>>>>>>> guide. >>>>>>>>>> >>>>>>>>>> Sandglass Foundation >>>>>>>>>> -------------------- >>>>>>>>>> >>>>>>>>>> Like David, I came to the conclusion that a framework rewrite >>>>>>>>>> would >>>>>>>>>> be >>>>>>>>>> easier outside the OFBiz community. So, I created my own library >>>>>>>>>> called >>>>>>>>>> Foundation: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> http://www.sandglass-software.com/products/sandglass/documents/Foundation_Brochure.pdf >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> (PDF) >>>>>>>>>> >>>>>>>>>> and I only mention it here to stress how wonderful it can be to >>>>>>>>>> start >>>>>>>>>> with >>>>>>>>>> a clean slate and design an API that is concise yet powerful. >>>>>>>>>> (Please >>>>>>>>>> do >>>>>>>>>> not discuss Foundation here, contact me privately if you want more >>>>>>>>>> information.) >>>>>>>>>> >>>>>>>>>> Some examples of what can be done with a rewrite: >>>>>>>>>> >>>>>>>>>> A single configuration file >>>>>>>>>> Use ANSI/ISO SQL SELECT statement strings instead of >>>>>>>>>> constructing >>>>>>>>>> complicated Java structures >>>>>>>>>> Simultaneous asynchronous queries >>>>>>>>>> Relational integrity across multiple datasources >>>>>>>>>> Multi-table SELECT across multiple datasources >>>>>>>>>> Automatic and transparent row version control >>>>>>>>>> Automatic and transparent multi-language datasource support >>>>>>>>>> Abstract entities (similar to SQL user types) >>>>>>>>>> Service engine throttling (protects against server >>>>>>>>>> over-utilization) >>>>>>>>>> Simplified security (authorization) ( >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> https://cwiki.apache.org/confluence/display/OFBIZ/OFBiz+Security+Redesign >>>>>>>>>> ) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Pure interface-based API - so developers are free to modify >>>>>>>>>> framework >>>>>>>>>> behavior by using decorators >>>>>>>>>> Thorough unit tests >>>>>>>>>> >>>>>>>>>> Benefits of a rewrite: >>>>>>>>>> >>>>>>>>>> Reduced resource requirements (lower hosting fees) >>>>>>>>>> Reduce application development time - due to a simplified >>>>>>>>>> API >>>>>>>>>> Easier framework code maintenance >>>>>>>>>> Better reliability >>>>>>>>>> >>>>>>>>>> Conclusion >>>>>>>>>> ---------- >>>>>>>>>> >>>>>>>>>> Like I said at the start, this is all I will say about the >>>>>>>>>> subject. >>>>>>>>>> I'm >>>>>>>>>> done trying to convince everyone. I hope someone agrees with me >>>>>>>>>> and >>>>>>>>>> they >>>>>>>>>> are able to build support for the idea. >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Adrian Crum >>>>>>>>>> Sandglass Software >>>>>>>>>> www.sandglass-software.com >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>> > |
Re: Why A Framework Rewrite Is Necessary
|
|
> On 15 Oct 2015, at 18:48, Scott Gray <[hidden email]> wrote: > > By the way, I'm surprised I haven't seen any efforts within the Moqui > community to rewrite any of the OFBiz apps. I don't have any ideas as to > why that hasn't happened, but I'm curious about it. Surely it would be > easier for the Moqui community who already know the framework and are > probably also familiar with OFBiz to try this than us. This I can comment on. The business level data model in Mantle Business Artifacts (separate from Moqui Framework) is very different from the one in OFBiz. It is not only the framework that has room for cleanup and enhancement, but the data model, services and UI as well. The data model and services in Mantle are now on par or more advanced that similar parts of OFBiz in many areas, though there are still various areas of business functionality supported in OFBiz that are not yet supported in Mantle services and such. Overall Mantle is being built in the context of very different contracts than early OFBiz, more ERP oriented in general, which is why certain things like payroll exist in Mantle but never were implemented in OFBiz. As strange as it may be (and really not my preference), the initial standards based in integrations in Mantle are for various EDI X12 messages as opposed to the OAGIS ones in OFBiz (I still like that standard MUCH more and plan to build around it eventually… but contracts do set priority, especially for larger efforts like that). The UI layer in the HiveMind (ERP and project management for service orgs) and POP Commerce (ERP and ecommerce for retail and wholesale) are also being built from scratch, and now in the hundreds of screens. The goal has always been to start from scratch with Moqui/Mantle/etc, based on many of the ideas and patterns that emerged over time in OFBiz. As for the application structure, and even screen designs, in OFBiz: they could be much better. Without customization and enhancement so many business processes are left hanging with no UI to complete them (though sometimes there are services in place). The menu structure and such is also little short of painful, and in all larger OFBiz based projects I’ve been on the application level has been redesigned and rewritten, to a lesser or greater extent reusing screens/forms/etc from the OOTB OFBiz apps. Even as open source contributors with limited time and resources, we can do better. OFBiz can be much more usable than it is now, the potential is there. In the mean time, I’ll admit there are other applications I take more inspiration from than OFBiz, on the UI level at least. On the data model and service/logic level there is a lot of inspiration to be taken from OFBiz, even if there is a lot of room for improvement (doing much more with much less code, and much more flexibly too). A framework rewrite may make things more efficient, easy, and clean in OFBiz, but there is a lot of room for improvement in the data model, logic, and UI layer implementations as well. Those sorts of changes can be done maintaining backward compatibility, and without starting over. The accounting in OFBiz is close to adequate to run a business on, but not there in various painful ways that I’ve been through with many clients. The order processing based on the ShoppingCart* objects is abysmal… inflexible and unnecessarily complex. What a revolution in OFBiz? Kill the ShoppingCart* objects, move all the logic to services, and keep cart state in the database instead of session objects. The menu structure in so many applications, and accounting may be the worst of them, is painful and confusing at best. I still see some of the bad decisions I made over a decade ago persisting there. Yes, I know these because they are my sins, my early mistakes… and I gave up on atoning for them as part of OFBiz. Whatever Adrian says, be careful of his comments and direction, he is a big part of the reason I gave up improving things in OFBiz itself. -David |
Re: Why A Framework Rewrite Is Necessary
|
|
In reply to this post by Adrian Crum-3
> On 15 Oct 2015, at 07:58, Adrian Crum <[hidden email]> wrote: > > Keep in mind that much of David's code in OFBiz has been rewritten. So yes, we CAN do a better job than him. I think there’s a name for this logical fallacy… > Also keep in mind that Moqui duplicates some of the problems I listed - so by using Moqui, we keep the same problems instead of fixing them. Could you be more specific, other than the type conversion stuff you mentioned many years ago (which I fully disagree with)? -David > On the other hand, it is David's responsibility to fix them, not ours. > > If we created a sub-project, then we would have the opportunity to review committer permissions and perhaps restrict access to the new code. > > Adrian Crum > Sandglass Software > www.sandglass-software.com > > On 10/15/2015 6:34 AM, Al Byers wrote: >> I was waiting for someone to bring this up. David Jones created Moqui with >> the same end in mind that the rewrite is meant to accomplish. Do you think >> that you will do a better job than him? Even if you could, would it be so >> much better that it warrants the effort that it would take? >> >> Is this a political thing? I don't know that David would give up control of >> Moqui Core and I, for one, would not want him to. So many of OFBiz's >> problems are the result of ineptitude on the part of committers who did not >> know what they were doing (like me.) But I don't know that the Core will >> change all that much going forward and there should be places in the Mantle >> to contribute and, most certainly, in the Crust. >> >> I know that there were some valid questions about licensing and project >> management, but I would think that they could be worked out. >> >> On Thu, Oct 15, 2015 at 4:46 AM, Hans Bakker <[hidden email]> >> wrote: >> >>> Why not skip this step, using moqui which is already up and running and >>> then as Adrian states as a next step: applications could be pulled down and >>> adapted to it. >>> >>> Hans >>> >>> >>> >>> On 15/10/15 16:51, Jacques Le Roux wrote: >>> >>>> I'm in the same mood than Paul and Scott. So a sub-project could indeed >>>> be a solution. >>>> >>>> Jacques >>>> >>>> Le 15/10/2015 03:11, Adrian Crum a écrit : >>>> >>>>> I agree that a sub-project would be nice to have. Once the new framework >>>>> is up and running, applications could be pulled down and adapted to it. >>>>> >>>>> Adrian Crum >>>>> Sandglass Software >>>>> www.sandglass-software.com >>>>> >>>>> On 10/14/2015 5:53 PM, Ron Wheeler wrote: >>>>> >>>>>> This seems to be very good advice. >>>>>> A completely separate sub-project under OFBiz with its own mailing lists >>>>>> would keep the people together yet all allow the new framework the >>>>>> flexibility to move forward. >>>>>> >>>>>> Ron >>>>>> On 14/10/2015 8:27 PM, Scott Gray wrote: >>>>>> >>>>>>> My advice is as follows: >>>>>>> 1. If people are interested, then get them together and start working >>>>>>> on it. >>>>>>> 2. Find somewhere to do the work. I don't think a branch is >>>>>>> appropriate >>>>>>> because it's completely new development rather than a refactoring. I >>>>>>> don't >>>>>>> have any objections to it being done under the ASF OFBiz umbrella >>>>>>> (although >>>>>>> I don't really see the need either). >>>>>>> 3. Set up a separate mailing list for discussion. Generally I'd try >>>>>>> and >>>>>>> keep quiet about it in the early stages on the dev/user lists or other >>>>>>> marketing channels because it could potentially harm adoption of our >>>>>>> existing framework (think AngularJS 2.0). >>>>>>> >>>>>>> There really isn't any need to get early stage sign-off from the PMC or >>>>>>> anyone else in the community. You only need enough PMC approval to >>>>>>> get the >>>>>>> required infrastructure sorted, which I don't think would be an issue. >>>>>>> >From there, it's really up to the community to decide whether or not >>>>>>> the >>>>>>> thing will fly. >>>>>>> >>>>>>> Regards >>>>>>> Scott >>>>>>> >>>>>>> >>>>>>> On 15 October 2015 at 08:21, Adrian Crum >>>>>>> <[hidden email] >>>>>>> >>>>>>>> wrote: >>>>>>>> I understand that Sharan brought up the framework rewrite subject at >>>>>>>> ApacheCon, and some attendees felt that the framework is fine and no >>>>>>>> action >>>>>>>> needs to be taken. >>>>>>>> >>>>>>>> In this message, I will try to give a detailed explanation of why a >>>>>>>> framework rewrite is necessary. I don't plan to take any further >>>>>>>> action on >>>>>>>> this subject, because I've brought it up before without success, and >>>>>>>> I'm >>>>>>>> tired of discussing it. It is my hope that the light bulb will click >>>>>>>> on in >>>>>>>> someone's head and they will take action. >>>>>>>> >>>>>>>> My Background >>>>>>>> ------------- >>>>>>>> >>>>>>>> I became a member of the OFBiz community in 2004. I immediately >>>>>>>> started >>>>>>>> making contributions to the project by supplying patches to the issue >>>>>>>> tracker. In 2007, I became a committer. Most of my initial work was >>>>>>>> on the >>>>>>>> UI and some work in the applications (mainly Asset Maintenance and >>>>>>>> Work >>>>>>>> Effort). I stayed away from touching the framework code because it was >>>>>>>> deep, dark, and scary. >>>>>>>> >>>>>>>> Eventually, I started to understand how the framework code works and I >>>>>>>> made some minor modifications. As my understanding grew, I progressed >>>>>>>> to >>>>>>>> rewriting large swaths of framework code - making it thread-safe, >>>>>>>> fault >>>>>>>> tolerant, efficient, and easier to use. >>>>>>>> >>>>>>>> I will list some of my contributions here, so everyone can have a >>>>>>>> clear >>>>>>>> understanding of my experience with the framework code: >>>>>>>> >>>>>>>> New Features >>>>>>>> >>>>>>>> User Preferences >>>>>>>> >>>>>>>> Visual Themes >>>>>>>> >>>>>>>> Custom UI Label XML File Format >>>>>>>> >>>>>>>> Temporal Expressions >>>>>>>> >>>>>>>> Data Type Conversion Framework >>>>>>>> >>>>>>>> Screen Widget Boundary Comments >>>>>>>> >>>>>>>> Metrics >>>>>>>> >>>>>>>> Integrations >>>>>>>> >>>>>>>> UEL >>>>>>>> >>>>>>>> iCalendar >>>>>>>> >>>>>>>> JSR 223 >>>>>>>> >>>>>>>> WebDAV >>>>>>>> >>>>>>>> LDAP >>>>>>>> >>>>>>>> Refactorings/Improvements >>>>>>>> >>>>>>>> FlexibleStringExpander >>>>>>>> >>>>>>>> FlexibleMapExpander >>>>>>>> >>>>>>>> FOP Integration >>>>>>>> >>>>>>>> FreeMarkerWorker >>>>>>>> >>>>>>>> Date-Time Handling >>>>>>>> >>>>>>>> Mini-language >>>>>>>> >>>>>>>> Job Scheduler >>>>>>>> >>>>>>>> In addition, I have performed innumerable framework bug fixes. >>>>>>>> >>>>>>>> So, the contents of this message come from years of experience mucking >>>>>>>> about in the framework code. >>>>>>>> >>>>>>>> Okay, let's get started... >>>>>>>> >>>>>>>> Initial Problem Statement >>>>>>>> ------------------------- >>>>>>>> >>>>>>>> In 2009, David Jones started a framework rewrite in a branch: >>>>>>>> >>>>>>>> >>>>>>>> https://svn.apache.org/repos/asf/ofbiz/branches/executioncontext20090716 >>>>>>>> >>>>>>>> At the time, there was some agreement that a rewrite was necessary, >>>>>>>> but >>>>>>>> there was disagreement as to how the rewrite should be incorporated >>>>>>>> into >>>>>>>> the project: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> https://mail-archives.apache.org/mod_mbox/ofbiz-dev/200908.mbox/%3C455601.62605.qm@...%3E >>>>>>>> >>>>>>>> >>>>>>>> There were concerns that a rewrite would break backward compatibility. >>>>>>>> Work on the rewrite branch stopped. Eventually, Jacopo suggested the >>>>>>>> community be more accepting of backward-incompatible changes: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> https://mail-archives.apache.org/mod_mbox/ofbiz-dev/201004.mbox/%3cD24F129D-4F9F-444E-84AF-ACA46F49990C@...%3e >>>>>>>> >>>>>>>> >>>>>>>> Despite an effort to convince David to proceed with the framework >>>>>>>> rewrite, >>>>>>>> he ended up doing it in a separate project: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> http://mail-archives.apache.org/mod_mbox/ofbiz-dev/201104.mbox/%3C07565C88-4023-4D24-93A3-A4906E86F939@...%3E >>>>>>>> >>>>>>>> >>>>>>>> This page describes differences between OFBiz and Moqui, and within >>>>>>>> it you >>>>>>>> can extract information on the problems David was trying to solve: >>>>>>>> >>>>>>>> http://sourceforge.net/p/moqui/discussion/1086127/thread/4c52f240/ >>>>>>>> >>>>>>>> There was an email he sent out on the OFBiz dev list where he listed >>>>>>>> the >>>>>>>> problems he saw in the framework, but I can't find it. The rest of >>>>>>>> this >>>>>>>> message will include the issues he mentioned (the ones I remember). I >>>>>>>> was >>>>>>>> in agreement with him at the time, and I still agree that a framework >>>>>>>> rewrite is necessary. >>>>>>>> >>>>>>>> The Problems >>>>>>>> ------------ >>>>>>>> >>>>>>>> Code is scattered everywhere - due to an initial effort to make the >>>>>>>> framework modular. This causes serious problems. The mere fact that >>>>>>>> components like entityext and securityext EXIST makes it clear that >>>>>>>> there >>>>>>>> are problems - those components should not be there. Also, we run >>>>>>>> into the >>>>>>>> recurring problem of circular dependencies (component A will not build >>>>>>>> unless component B is built, and component B will not build unless >>>>>>>> component A is built). >>>>>>>> >>>>>>>> Bad separation of concerns. There are far too many examples of classes >>>>>>>> that try to be everything to everyone. This makes debugging >>>>>>>> difficult, and >>>>>>>> it makes maintenance/improvements a nightmare. [Using an analogy, >>>>>>>> consider >>>>>>>> an automobile design where a spark plug is not separate from a >>>>>>>> transmission. Instead, the automobile uses a spark-plug-transmission. >>>>>>>> So >>>>>>>> when the engine is running rough because the spark plug is bad, you >>>>>>>> have to >>>>>>>> replace the spark plug AND the transmission.] A good framework >>>>>>>> example can >>>>>>>> be found in my rewrite of the mini-language code. Originally, the >>>>>>>> models >>>>>>>> AND the script execution context both contained script behaviors - >>>>>>>> making >>>>>>>> debugging/improvements difficult. I changed it so only the models >>>>>>>> contain >>>>>>>> script behavior and the script execution context contains only the >>>>>>>> script >>>>>>>> execution state. >>>>>>>> >>>>>>>> Lack of good OO design. There are many places where a bit of framework >>>>>>>> functionality is contained in a single method that is hundreds or >>>>>>>> thousands >>>>>>>> of lines long. There is a term for that: Brittle Code. Code isn't >>>>>>>> reused. >>>>>>>> Instead, it is copy-and-pasted all over - so when a problem is found >>>>>>>> in the >>>>>>>> C&P code, it has to be fixed in many places instead of one. >>>>>>>> >>>>>>>> Fail-slow design. There are a lot of places in low-level code where an >>>>>>>> error condition is encountered, but instead of throwing an exception, >>>>>>>> the >>>>>>>> error is ignored and maybe it is logged, or the code tries to "guess" >>>>>>>> at a >>>>>>>> solution and then provide an arbitrary default behavior. I've seen >>>>>>>> many >>>>>>>> developers struggle with debugging a problem because they didn't look >>>>>>>> at >>>>>>>> the logs, or because the error was not logged and there is no way of >>>>>>>> knowing what caused it. They end up spending hours single-stepping >>>>>>>> through >>>>>>>> code until it reaches the error. >>>>>>>> >>>>>>>> Out-of-date code. A good example is the use of Javolution. That >>>>>>>> library >>>>>>>> was beneficial in the Java 1.4 days, but it is not necessary today >>>>>>>> because >>>>>>>> of improved garbage collection. Another good example is DCL code. DCL >>>>>>>> was >>>>>>>> used extensively in OFBiz, but it is clearly documented to be an >>>>>>>> unreliable >>>>>>>> design (I can get it to fail 90% of the time). Some DCL code has been >>>>>>>> replaced, but a lot of it is still there. >>>>>>>> >>>>>>>> Portions of the API are overly complicated. Some methods require a >>>>>>>> collection of user-specified artifacts/arguments, which makes client >>>>>>>> code >>>>>>>> complicated and verbose. (David solved that problem with his Execution >>>>>>>> Context.) Portions of the API are cluttered with unnecessary >>>>>>>> "convenience >>>>>>>> methods" - making the API harder to learn and memorize. In some >>>>>>>> places, a >>>>>>>> domain-specific API is spread across instance methods and static >>>>>>>> methods >>>>>>>> and across different classes - making the API hard to understand and >>>>>>>> use. >>>>>>>> Yes, there can be good designs that require something like that, but >>>>>>>> in the >>>>>>>> OFBiz framework, it exists because of a bad design, not a good one. >>>>>>>> >>>>>>>> Use of thread-local variables. This makes multi-threaded design >>>>>>>> impossible. The J2EE specification and the Servlet API require one >>>>>>>> thread >>>>>>>> per request (and most J2EE libraries depend on that behavior), so the >>>>>>>> current design makes sense from a J2EE perspective, but what if I >>>>>>>> don't >>>>>>>> want to run the framework in a J2EE container? Which leads to the next >>>>>>>> problem... >>>>>>>> >>>>>>>> Dependence on J2EE designs/APIs/libraries. There are developers in the >>>>>>>> Java community (myself included) who are beginning to question if >>>>>>>> J2EE is >>>>>>>> really necessary to run web applications. The folks at Atomikos are a >>>>>>>> good >>>>>>>> example. OFBiz does not use EJBs, so tying the framework to J2EE does >>>>>>>> not >>>>>>>> make sense. It would be better if the framework was designed to run >>>>>>>> outside >>>>>>>> a J2EE container, and then have container integration as an option. >>>>>>>> >>>>>>>> Configuration files are scattered everywhere. Anyone who has deployed >>>>>>>> OFBiz in a production environment will agree this is a problem. Try >>>>>>>> changing the HTTP/HTTPS and port settings - it is a nightmare. Some >>>>>>>> configuration settings are in nonsensical places. >>>>>>>> >>>>>>>> An abysmal lack of unit testing. I don't have an exact figure for code >>>>>>>> coverage, but my gut feeling is coverage is less than 10%. Basically, >>>>>>>> we >>>>>>>> all have our fingers crossed - hoping that the framework code works as >>>>>>>> expected. This was made painfully obvious a while back when I was >>>>>>>> looking >>>>>>>> at some entity caching code and thought to myself "this code can't >>>>>>>> work." >>>>>>>> So I wrote some entity cache unit tests and confirmed that the entity >>>>>>>> cache >>>>>>>> had serious problems. Think about that - years passed with no entity >>>>>>>> cache >>>>>>>> unit tests and consequently we had no idea it wasn't working. >>>>>>>> >>>>>>>> Fix Versus Rewrite >>>>>>>> ------------------ >>>>>>>> >>>>>>>> Jira issues could be created for these problems and teams of >>>>>>>> developers >>>>>>>> could work to fix them. >>>>>>>> >>>>>>>> Or, we could create a branch and start over from scratch. This time >>>>>>>> around, there should be less push-back from people concerned about >>>>>>>> backwards compatibility. A rewrite offers the advantage of >>>>>>>> reconsidering >>>>>>>> everything - like API design, general problem solving, and new >>>>>>>> features. >>>>>>>> >>>>>>>> I created a Wiki page for a framework design: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> https://cwiki.apache.org/confluence/display/OFBADMIN/Another+Framework+Vision >>>>>>>> >>>>>>>> >>>>>>>> but there hasn't been much interest in it. If the community decides >>>>>>>> to go >>>>>>>> ahead with a rewrite, then please feel free to use the Wiki pages as a >>>>>>>> guide. >>>>>>>> >>>>>>>> Sandglass Foundation >>>>>>>> -------------------- >>>>>>>> >>>>>>>> Like David, I came to the conclusion that a framework rewrite would be >>>>>>>> easier outside the OFBiz community. So, I created my own library >>>>>>>> called >>>>>>>> Foundation: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> http://www.sandglass-software.com/products/sandglass/documents/Foundation_Brochure.pdf >>>>>>>> >>>>>>>> >>>>>>>> (PDF) >>>>>>>> >>>>>>>> and I only mention it here to stress how wonderful it can be to start >>>>>>>> with >>>>>>>> a clean slate and design an API that is concise yet powerful. (Please >>>>>>>> do >>>>>>>> not discuss Foundation here, contact me privately if you want more >>>>>>>> information.) >>>>>>>> >>>>>>>> Some examples of what can be done with a rewrite: >>>>>>>> >>>>>>>> A single configuration file >>>>>>>> Use ANSI/ISO SQL SELECT statement strings instead of constructing >>>>>>>> complicated Java structures >>>>>>>> Simultaneous asynchronous queries >>>>>>>> Relational integrity across multiple datasources >>>>>>>> Multi-table SELECT across multiple datasources >>>>>>>> Automatic and transparent row version control >>>>>>>> Automatic and transparent multi-language datasource support >>>>>>>> Abstract entities (similar to SQL user types) >>>>>>>> Service engine throttling (protects against server >>>>>>>> over-utilization) >>>>>>>> Simplified security (authorization) ( >>>>>>>> >>>>>>>> https://cwiki.apache.org/confluence/display/OFBIZ/OFBiz+Security+Redesign) >>>>>>>> >>>>>>>> >>>>>>>> Pure interface-based API - so developers are free to modify >>>>>>>> framework >>>>>>>> behavior by using decorators >>>>>>>> Thorough unit tests >>>>>>>> >>>>>>>> Benefits of a rewrite: >>>>>>>> >>>>>>>> Reduced resource requirements (lower hosting fees) >>>>>>>> Reduce application development time - due to a simplified API >>>>>>>> Easier framework code maintenance >>>>>>>> Better reliability >>>>>>>> >>>>>>>> Conclusion >>>>>>>> ---------- >>>>>>>> >>>>>>>> Like I said at the start, this is all I will say about the subject. >>>>>>>> I'm >>>>>>>> done trying to convince everyone. I hope someone agrees with me and >>>>>>>> they >>>>>>>> are able to build support for the idea. >>>>>>>> >>>>>>>> -- >>>>>>>> Adrian Crum >>>>>>>> Sandglass Software >>>>>>>> www.sandglass-software.com >>>>>>>> >>>>>>>> >>>>>> >>>>>> >>>>> >>> >> |
Re: Why A Framework Rewrite Is Necessary
|
|
In reply to this post by David E. Jones-2
Nice shot, David. EDI X12 is important to build a system like Uber. I'm sure this will be spread in many industries. So personally I think it's beyond OFBiz's target market(ERP/CRM).
-----邮件原件----- 发件人: David E. Jones [mailto:[hidden email]] 发送时间: 2015年10月16日 10:58 收件人: [hidden email] 主题: Re: Why A Framework Rewrite Is Necessary > On 15 Oct 2015, at 18:48, Scott Gray <[hidden email]> wrote: > > By the way, I'm surprised I haven't seen any efforts within the Moqui > community to rewrite any of the OFBiz apps. I don't have any ideas as to > why that hasn't happened, but I'm curious about it. Surely it would be > easier for the Moqui community who already know the framework and are > probably also familiar with OFBiz to try this than us. This I can comment on. The business level data model in Mantle Business Artifacts (separate from Moqui Framework) is very different from the one in OFBiz. It is not only the framework that has room for cleanup and enhancement, but the data model, services and UI as well. The data model and services in Mantle are now on par or more advanced that similar parts of OFBiz in many areas, though there are still various areas of business functionality supported in OFBiz that are not yet supported in Mantle services and such. Overall Mantle is being built in the context of very different contracts than early OFBiz, more ERP oriented in general, which is why certain things like payroll exist in Mantle but never were implemented in OFBiz. As strange as it may be (and really not my preference), the initial standards based in integrations in Mantle are for various EDI X12 messages as opposed to the OAGIS ones in OFBiz (I still like that standard MUCH more and plan to build around it eventually… but contracts do set priority, especially for larger efforts like that). The UI layer in the HiveMind (ERP and project management for service orgs) and POP Commerce (ERP and ecommerce for retail and wholesale) are also being built from scratch, and now in the hundreds of screens. The goal has always been to start from scratch with Moqui/Mantle/etc, based on many of the ideas and patterns that emerged over time in OFBiz. As for the application structure, and even screen designs, in OFBiz: they could be much better. Without customization and enhancement so many business processes are left hanging with no UI to complete them (though sometimes there are services in place). The menu structure and such is also little short of painful, and in all larger OFBiz based projects I’ve been on the application level has been redesigned and rewritten, to a lesser or greater extent reusing screens/forms/etc from the OOTB OFBiz apps. Even as open source contributors with limited time and resources, we can do better. OFBiz can be much more usable than it is now, the potential is there. In the mean time, I’ll admit there are other applications I take more inspiration from than OFBiz, on the UI level at least. On the data model and service/logic level there is a lot of inspiration to be taken from OFBiz, even if there is a lot of room for improvement (doing much more with much less code, and much more flexibly too). A framework rewrite may make things more efficient, easy, and clean in OFBiz, but there is a lot of room for improvement in the data model, logic, and UI layer implementations as well. Those sorts of changes can be done maintaining backward compatibility, and without starting over. The accounting in OFBiz is close to adequate to run a business on, but not there in various painful ways that I’ve been through with many clients. The order processing based on the ShoppingCart* objects is abysmal… inflexible and unnecessarily complex. What a revolution in OFBiz? Kill the ShoppingCart* objects, move all the logic to services, and keep cart state in the database instead of session objects. The menu structure in so many applications, and accounting may be the worst of them, is painful and confusing at best. I still see some of the bad decisions I made over a decade ago persisting there. Yes, I know these because they are my sins, my early mistakes… and I gave up on atoning for them as part of OFBiz. Whatever Adrian says, be careful of his comments and direction, he is a big part of the reason I gave up improving things in OFBiz itself. -David |
Re: Why A Framework Rewrite Is Necessary
|
|
In reply to this post by David E. Jones-2
On Fri, Oct 16, 2015 at 5:31 AM, David E. Jones <[hidden email]> wrote:
> > > > On 15 Oct 2015, at 07:58, Adrian Crum < > [hidden email]> wrote: > > > > Keep in mind that much of David's code in OFBiz has been rewritten. So > yes, we CAN do a better job than him. > > I think there’s a name for this logical fallacy… > > And this could also be called a logical fallacy. But let us not make it opposing viewpoints on this. > > Also keep in mind that Moqui duplicates some of the problems I listed - > so by using Moqui, we keep the same problems instead of fixing them. > > Could you be more specific, other than the type conversion stuff you > mentioned many years ago (which I fully disagree with)? > > This is not about who is right or wrong, but where the community wants to go. I understand the reluctance of the community, because the impact will be huge. When looking at the data in OpenHub I see OFBiz having an estimate effort spend of 519 person years vs 6 for the combined Moqui-Mantle-HiveMindPM-PopCommerce suite. And one of the reasons behind it is simple: Many more have worked on OFBiz (from day 1) than on the Moqui suite. One could even argue that both directions took the same number of years in duration to get where they are now. Without all the experiences regarding the OFBiz product there couldn't have been an evolution called the Moqui suite. Coming back to opting for a new direction, as Scott has stated we can have this in a separate code repository (subproject, like many other Apache projects do their work) even combined with a new JIRA an Wiki under the umbrella of the OFBiz project. Based on the comments provided, this seems like a logical choice to ensure that adopters of the current solution will keep the support of the community while at the same time ensuring containment of the new approach. But these are mere technical, supportive aspects. The more important issue is what this new solution will encompass. There are talks of a rewrite, which sounds like reinvention of the wheel. But I guess it is not like that. Yet, taking decisions based on a few one-pagers (e.g. http://www.sandglass-software.com/products/sandglass/documents/Foundation_Brochure.pdf) are seldom done. Maybe it works for a single person, but I doubt it will make a community fly. Whether fix or rewrite, choices will be made regarding the supporting 3rd party libraries/solutions and the community needs to understand the impact to get behind it. So before we embark on the coding trip, let us get into trying to define a bit more what the new solution will encompass and get consensus on that. Another issue regarding getting the community behind behind this new effort is this: 'restrict access to the new code'. I guess this meant restrict write access. Though understandable from a avoidance of dilution/scope creep point of view, I see this as a wrong direction. This 'proposed' exclusion of contributors of the kind will bring us back to where this project came from: discrimination and favouritism. I even doubt that this is possible under the current principles of the ASF. Given that this is an enormous undertaking, we need to get as many on board as possible. Not only to ensure that the decisions (at each level) will get consensus, but also the ensure that every aspect down the line will get addressed (e.g. documentation, test definitions, marketing/promotions, etc) in order to get this kite flying. Best regards, Pierre Smits *OFBiz Extensions Marketplace* http://oem.ofbizci.net/oci-2/ |
Re: Why A Framework Rewrite Is Necessary
Administrator
|
In reply to this post by Scott Gray-3
I have not much time to comment at the moment, but at large this seems wise to me (in other words I kinda agree)
Jacques Le 16/10/2015 02:48, Scott Gray a écrit : > I don't think this community is large enough to attempt to rewrite all of > our applications to suit Moqui while at the same time still supporting them > under the existing framework. That may well be the case for a homegrown > framework rewrite as well depending on how the applications would be > affected. This is the main reason why I tend to prefer incremental > improvements to the existing framework. But if people within the community > wanted to try for either of the "new framework" options, I certainly don't > have a problem with it and am happy to see some infrastructure donated to > the cause. > > There's simply no need to form a consensus on any major decision in the > early stages, proponents just need to be willing to accept that their > endeavors might not become part of the project. David certainly didn't > wait until he had a community backing him to go ahead and start work on > Moqui. The best way to get something done is to get started on it, if no > one follows then it's either a bad idea or you're doing a bad job of > selling it. > > By the way, I'm surprised I haven't seen any efforts within the Moqui > community to rewrite any of the OFBiz apps. I don't have any ideas as to > why that hasn't happened, but I'm curious about it. Surely it would be > easier for the Moqui community who already know the framework and are > probably also familiar with OFBiz to try this than us. > > Regards > Scott > > > On 16 October 2015 at 12:20, Hans Bakker <[hidden email]> wrote: > >> Well said: +1 >> >> >> On 15/10/15 22:13, Al Byers wrote: >> >>> Fair enough, Adrian, but is this a widely held position on OFBiz that as a >>> group you could produce a product that is so much better than Moqui that >>> it >>> would be worth the time and risk (the risk of burning cycles that you will >>> never get back only to have the venture never succeed.) Have all the >>> people >>> who would like to do a rewrite done the same gap analysis on Moqui vs what >>> you would like to see OFBiz be? Does everyone understand and agree that >>> the >>> deficiencies that you see in Moqui are important? My concern is that the >>> many people who understandably want a rewrite of OFBiz will be swayed by >>> your arguments without there being due diligence being done. I think that >>> there are a lot of veterans of OFBiz who have confidence in David's >>> development skills, but who are not pressing for a move to Moqui or for a >>> rewrite because they have so much invested in the current state of OFBiz >>> and recognized what a long shot a rewrite would be and how much work it >>> would require of them to move over. But anybody who is open to a rewrite >>> would be foolish not to consider the work done by the principal architect >>> of OFBiz; work that was meant to address deficiencies in OFBiz. >>> >>> On Thu, Oct 15, 2015 at 7:58 AM, Adrian Crum < >>> [hidden email]> wrote: >>> >>> Keep in mind that much of David's code in OFBiz has been rewritten. So >>>> yes, we CAN do a better job than him. Also keep in mind that Moqui >>>> duplicates some of the problems I listed - so by using Moqui, we keep the >>>> same problems instead of fixing them. On the other hand, it is David's >>>> responsibility to fix them, not ours. >>>> >>>> If we created a sub-project, then we would have the opportunity to review >>>> committer permissions and perhaps restrict access to the new code. >>>> >>>> Adrian Crum >>>> Sandglass Software >>>> www.sandglass-software.com >>>> >>>> On 10/15/2015 6:34 AM, Al Byers wrote: >>>> >>>> I was waiting for someone to bring this up. David Jones created Moqui >>>>> with >>>>> the same end in mind that the rewrite is meant to accomplish. Do you >>>>> think >>>>> that you will do a better job than him? Even if you could, would it be >>>>> so >>>>> much better that it warrants the effort that it would take? >>>>> >>>>> Is this a political thing? I don't know that David would give up control >>>>> of >>>>> Moqui Core and I, for one, would not want him to. So many of OFBiz's >>>>> problems are the result of ineptitude on the part of committers who did >>>>> not >>>>> know what they were doing (like me.) But I don't know that the Core will >>>>> change all that much going forward and there should be places in the >>>>> Mantle >>>>> to contribute and, most certainly, in the Crust. >>>>> >>>>> I know that there were some valid questions about licensing and project >>>>> management, but I would think that they could be worked out. >>>>> >>>>> On Thu, Oct 15, 2015 at 4:46 AM, Hans Bakker < >>>>> [hidden email]> >>>>> wrote: >>>>> >>>>> Why not skip this step, using moqui which is already up and running and >>>>> >>>>>> then as Adrian states as a next step: applications could be pulled down >>>>>> and >>>>>> adapted to it. >>>>>> >>>>>> Hans >>>>>> >>>>>> >>>>>> >>>>>> On 15/10/15 16:51, Jacques Le Roux wrote: >>>>>> >>>>>> I'm in the same mood than Paul and Scott. So a sub-project could indeed >>>>>> >>>>>>> be a solution. >>>>>>> >>>>>>> Jacques >>>>>>> >>>>>>> Le 15/10/2015 03:11, Adrian Crum a écrit : >>>>>>> >>>>>>> I agree that a sub-project would be nice to have. Once the new >>>>>>> framework >>>>>>> >>>>>>>> is up and running, applications could be pulled down and adapted to >>>>>>>> it. >>>>>>>> >>>>>>>> Adrian Crum >>>>>>>> Sandglass Software >>>>>>>> www.sandglass-software.com >>>>>>>> >>>>>>>> On 10/14/2015 5:53 PM, Ron Wheeler wrote: >>>>>>>> >>>>>>>> This seems to be very good advice. >>>>>>>> >>>>>>>>> A completely separate sub-project under OFBiz with its own mailing >>>>>>>>> lists >>>>>>>>> would keep the people together yet all allow the new framework the >>>>>>>>> flexibility to move forward. >>>>>>>>> >>>>>>>>> Ron >>>>>>>>> On 14/10/2015 8:27 PM, Scott Gray wrote: >>>>>>>>> >>>>>>>>> My advice is as follows: >>>>>>>>> >>>>>>>>>> 1. If people are interested, then get them together and start >>>>>>>>>> working >>>>>>>>>> on it. >>>>>>>>>> 2. Find somewhere to do the work. I don't think a branch is >>>>>>>>>> appropriate >>>>>>>>>> because it's completely new development rather than a >>>>>>>>>> refactoring. I >>>>>>>>>> don't >>>>>>>>>> have any objections to it being done under the ASF OFBiz umbrella >>>>>>>>>> (although >>>>>>>>>> I don't really see the need either). >>>>>>>>>> 3. Set up a separate mailing list for discussion. Generally I'd >>>>>>>>>> try >>>>>>>>>> and >>>>>>>>>> keep quiet about it in the early stages on the dev/user lists or >>>>>>>>>> other >>>>>>>>>> marketing channels because it could potentially harm adoption of >>>>>>>>>> our >>>>>>>>>> existing framework (think AngularJS 2.0). >>>>>>>>>> >>>>>>>>>> There really isn't any need to get early stage sign-off from the >>>>>>>>>> PMC >>>>>>>>>> or >>>>>>>>>> anyone else in the community. You only need enough PMC approval to >>>>>>>>>> get the >>>>>>>>>> required infrastructure sorted, which I don't think would be an >>>>>>>>>> issue. >>>>>>>>>> >From there, it's really up to the community to decide whether or >>>>>>>>>> not >>>>>>>>>> the >>>>>>>>>> thing will fly. >>>>>>>>>> >>>>>>>>>> Regards >>>>>>>>>> Scott >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 15 October 2015 at 08:21, Adrian Crum >>>>>>>>>> <[hidden email] >>>>>>>>>> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> I understand that Sharan brought up the framework rewrite subject >>>>>>>>>>> at >>>>>>>>>>> ApacheCon, and some attendees felt that the framework is fine and >>>>>>>>>>> no >>>>>>>>>>> action >>>>>>>>>>> needs to be taken. >>>>>>>>>>> >>>>>>>>>>> In this message, I will try to give a detailed explanation of why >>>>>>>>>>> a >>>>>>>>>>> framework rewrite is necessary. I don't plan to take any further >>>>>>>>>>> action on >>>>>>>>>>> this subject, because I've brought it up before without success, >>>>>>>>>>> and >>>>>>>>>>> I'm >>>>>>>>>>> tired of discussing it. It is my hope that the light bulb will >>>>>>>>>>> click >>>>>>>>>>> on in >>>>>>>>>>> someone's head and they will take action. >>>>>>>>>>> >>>>>>>>>>> My Background >>>>>>>>>>> ------------- >>>>>>>>>>> >>>>>>>>>>> I became a member of the OFBiz community in 2004. I immediately >>>>>>>>>>> started >>>>>>>>>>> making contributions to the project by supplying patches to the >>>>>>>>>>> issue >>>>>>>>>>> tracker. In 2007, I became a committer. Most of my initial work >>>>>>>>>>> was >>>>>>>>>>> on the >>>>>>>>>>> UI and some work in the applications (mainly Asset Maintenance and >>>>>>>>>>> Work >>>>>>>>>>> Effort). I stayed away from touching the framework code because it >>>>>>>>>>> was >>>>>>>>>>> deep, dark, and scary. >>>>>>>>>>> >>>>>>>>>>> Eventually, I started to understand how the framework code works >>>>>>>>>>> and I >>>>>>>>>>> made some minor modifications. As my understanding grew, I >>>>>>>>>>> progressed >>>>>>>>>>> to >>>>>>>>>>> rewriting large swaths of framework code - making it thread-safe, >>>>>>>>>>> fault >>>>>>>>>>> tolerant, efficient, and easier to use. >>>>>>>>>>> >>>>>>>>>>> I will list some of my contributions here, so everyone can have a >>>>>>>>>>> clear >>>>>>>>>>> understanding of my experience with the framework code: >>>>>>>>>>> >>>>>>>>>>> New Features >>>>>>>>>>> >>>>>>>>>>> User Preferences >>>>>>>>>>> >>>>>>>>>>> Visual Themes >>>>>>>>>>> >>>>>>>>>>> Custom UI Label XML File Format >>>>>>>>>>> >>>>>>>>>>> Temporal Expressions >>>>>>>>>>> >>>>>>>>>>> Data Type Conversion Framework >>>>>>>>>>> >>>>>>>>>>> Screen Widget Boundary Comments >>>>>>>>>>> >>>>>>>>>>> Metrics >>>>>>>>>>> >>>>>>>>>>> Integrations >>>>>>>>>>> >>>>>>>>>>> UEL >>>>>>>>>>> >>>>>>>>>>> iCalendar >>>>>>>>>>> >>>>>>>>>>> JSR 223 >>>>>>>>>>> >>>>>>>>>>> WebDAV >>>>>>>>>>> >>>>>>>>>>> LDAP >>>>>>>>>>> >>>>>>>>>>> Refactorings/Improvements >>>>>>>>>>> >>>>>>>>>>> FlexibleStringExpander >>>>>>>>>>> >>>>>>>>>>> FlexibleMapExpander >>>>>>>>>>> >>>>>>>>>>> FOP Integration >>>>>>>>>>> >>>>>>>>>>> FreeMarkerWorker >>>>>>>>>>> >>>>>>>>>>> Date-Time Handling >>>>>>>>>>> >>>>>>>>>>> Mini-language >>>>>>>>>>> >>>>>>>>>>> Job Scheduler >>>>>>>>>>> >>>>>>>>>>> In addition, I have performed innumerable framework bug fixes. >>>>>>>>>>> >>>>>>>>>>> So, the contents of this message come from years of experience >>>>>>>>>>> mucking >>>>>>>>>>> about in the framework code. >>>>>>>>>>> >>>>>>>>>>> Okay, let's get started... >>>>>>>>>>> >>>>>>>>>>> Initial Problem Statement >>>>>>>>>>> ------------------------- >>>>>>>>>>> >>>>>>>>>>> In 2009, David Jones started a framework rewrite in a branch: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> https://svn.apache.org/repos/asf/ofbiz/branches/executioncontext20090716 >>>>>>>>>>> >>>>>>>>>>> At the time, there was some agreement that a rewrite was >>>>>>>>>>> necessary, >>>>>>>>>>> but >>>>>>>>>>> there was disagreement as to how the rewrite should be >>>>>>>>>>> incorporated >>>>>>>>>>> into >>>>>>>>>>> the project: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> https://mail-archives.apache.org/mod_mbox/ofbiz-dev/200908.mbox/%3C455601.62605.qm@...%3E >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> There were concerns that a rewrite would break backward >>>>>>>>>>> compatibility. >>>>>>>>>>> Work on the rewrite branch stopped. Eventually, Jacopo suggested >>>>>>>>>>> the >>>>>>>>>>> community be more accepting of backward-incompatible changes: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> https://mail-archives.apache.org/mod_mbox/ofbiz-dev/201004.mbox/%3cD24F129D-4F9F-444E-84AF-ACA46F49990C@...%3e >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Despite an effort to convince David to proceed with the framework >>>>>>>>>>> rewrite, >>>>>>>>>>> he ended up doing it in a separate project: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> http://mail-archives.apache.org/mod_mbox/ofbiz-dev/201104.mbox/%3C07565C88-4023-4D24-93A3-A4906E86F939@...%3E >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> This page describes differences between OFBiz and Moqui, and >>>>>>>>>>> within >>>>>>>>>>> it you >>>>>>>>>>> can extract information on the problems David was trying to solve: >>>>>>>>>>> >>>>>>>>>>> http://sourceforge.net/p/moqui/discussion/1086127/thread/4c52f240/ >>>>>>>>>>> >>>>>>>>>>> There was an email he sent out on the OFBiz dev list where he >>>>>>>>>>> listed >>>>>>>>>>> the >>>>>>>>>>> problems he saw in the framework, but I can't find it. The rest of >>>>>>>>>>> this >>>>>>>>>>> message will include the issues he mentioned (the ones I >>>>>>>>>>> remember). >>>>>>>>>>> I >>>>>>>>>>> was >>>>>>>>>>> in agreement with him at the time, and I still agree that a >>>>>>>>>>> framework >>>>>>>>>>> rewrite is necessary. >>>>>>>>>>> >>>>>>>>>>> The Problems >>>>>>>>>>> ------------ >>>>>>>>>>> >>>>>>>>>>> Code is scattered everywhere - due to an initial effort to make >>>>>>>>>>> the >>>>>>>>>>> framework modular. This causes serious problems. The mere fact >>>>>>>>>>> that >>>>>>>>>>> components like entityext and securityext EXIST makes it clear >>>>>>>>>>> that >>>>>>>>>>> there >>>>>>>>>>> are problems - those components should not be there. Also, we run >>>>>>>>>>> into the >>>>>>>>>>> recurring problem of circular dependencies (component A will not >>>>>>>>>>> build >>>>>>>>>>> unless component B is built, and component B will not build unless >>>>>>>>>>> component A is built). >>>>>>>>>>> >>>>>>>>>>> Bad separation of concerns. There are far too many examples of >>>>>>>>>>> classes >>>>>>>>>>> that try to be everything to everyone. This makes debugging >>>>>>>>>>> difficult, and >>>>>>>>>>> it makes maintenance/improvements a nightmare. [Using an analogy, >>>>>>>>>>> consider >>>>>>>>>>> an automobile design where a spark plug is not separate from a >>>>>>>>>>> transmission. Instead, the automobile uses a >>>>>>>>>>> spark-plug-transmission. >>>>>>>>>>> So >>>>>>>>>>> when the engine is running rough because the spark plug is bad, >>>>>>>>>>> you >>>>>>>>>>> have to >>>>>>>>>>> replace the spark plug AND the transmission.] A good framework >>>>>>>>>>> example can >>>>>>>>>>> be found in my rewrite of the mini-language code. Originally, the >>>>>>>>>>> models >>>>>>>>>>> AND the script execution context both contained script behaviors - >>>>>>>>>>> making >>>>>>>>>>> debugging/improvements difficult. I changed it so only the models >>>>>>>>>>> contain >>>>>>>>>>> script behavior and the script execution context contains only the >>>>>>>>>>> script >>>>>>>>>>> execution state. >>>>>>>>>>> >>>>>>>>>>> Lack of good OO design. There are many places where a bit of >>>>>>>>>>> framework >>>>>>>>>>> functionality is contained in a single method that is hundreds or >>>>>>>>>>> thousands >>>>>>>>>>> of lines long. There is a term for that: Brittle Code. Code isn't >>>>>>>>>>> reused. >>>>>>>>>>> Instead, it is copy-and-pasted all over - so when a problem is >>>>>>>>>>> found >>>>>>>>>>> in the >>>>>>>>>>> C&P code, it has to be fixed in many places instead of one. >>>>>>>>>>> >>>>>>>>>>> Fail-slow design. There are a lot of places in low-level code >>>>>>>>>>> where >>>>>>>>>>> an >>>>>>>>>>> error condition is encountered, but instead of throwing an >>>>>>>>>>> exception, >>>>>>>>>>> the >>>>>>>>>>> error is ignored and maybe it is logged, or the code tries to >>>>>>>>>>> "guess" >>>>>>>>>>> at a >>>>>>>>>>> solution and then provide an arbitrary default behavior. I've seen >>>>>>>>>>> many >>>>>>>>>>> developers struggle with debugging a problem because they didn't >>>>>>>>>>> look >>>>>>>>>>> at >>>>>>>>>>> the logs, or because the error was not logged and there is no way >>>>>>>>>>> of >>>>>>>>>>> knowing what caused it. They end up spending hours single-stepping >>>>>>>>>>> through >>>>>>>>>>> code until it reaches the error. >>>>>>>>>>> >>>>>>>>>>> Out-of-date code. A good example is the use of Javolution. That >>>>>>>>>>> library >>>>>>>>>>> was beneficial in the Java 1.4 days, but it is not necessary today >>>>>>>>>>> because >>>>>>>>>>> of improved garbage collection. Another good example is DCL code. >>>>>>>>>>> DCL >>>>>>>>>>> was >>>>>>>>>>> used extensively in OFBiz, but it is clearly documented to be an >>>>>>>>>>> unreliable >>>>>>>>>>> design (I can get it to fail 90% of the time). Some DCL code has >>>>>>>>>>> been >>>>>>>>>>> replaced, but a lot of it is still there. >>>>>>>>>>> >>>>>>>>>>> Portions of the API are overly complicated. Some methods require a >>>>>>>>>>> collection of user-specified artifacts/arguments, which makes >>>>>>>>>>> client >>>>>>>>>>> code >>>>>>>>>>> complicated and verbose. (David solved that problem with his >>>>>>>>>>> Execution >>>>>>>>>>> Context.) Portions of the API are cluttered with unnecessary >>>>>>>>>>> "convenience >>>>>>>>>>> methods" - making the API harder to learn and memorize. In some >>>>>>>>>>> places, a >>>>>>>>>>> domain-specific API is spread across instance methods and static >>>>>>>>>>> methods >>>>>>>>>>> and across different classes - making the API hard to understand >>>>>>>>>>> and >>>>>>>>>>> use. >>>>>>>>>>> Yes, there can be good designs that require something like that, >>>>>>>>>>> but >>>>>>>>>>> in the >>>>>>>>>>> OFBiz framework, it exists because of a bad design, not a good >>>>>>>>>>> one. >>>>>>>>>>> >>>>>>>>>>> Use of thread-local variables. This makes multi-threaded design >>>>>>>>>>> impossible. The J2EE specification and the Servlet API require one >>>>>>>>>>> thread >>>>>>>>>>> per request (and most J2EE libraries depend on that behavior), so >>>>>>>>>>> the >>>>>>>>>>> current design makes sense from a J2EE perspective, but what if I >>>>>>>>>>> don't >>>>>>>>>>> want to run the framework in a J2EE container? Which leads to the >>>>>>>>>>> next >>>>>>>>>>> problem... >>>>>>>>>>> >>>>>>>>>>> Dependence on J2EE designs/APIs/libraries. There are developers in >>>>>>>>>>> the >>>>>>>>>>> Java community (myself included) who are beginning to question if >>>>>>>>>>> J2EE is >>>>>>>>>>> really necessary to run web applications. The folks at Atomikos >>>>>>>>>>> are >>>>>>>>>>> a >>>>>>>>>>> good >>>>>>>>>>> example. OFBiz does not use EJBs, so tying the framework to J2EE >>>>>>>>>>> does >>>>>>>>>>> not >>>>>>>>>>> make sense. It would be better if the framework was designed to >>>>>>>>>>> run >>>>>>>>>>> outside >>>>>>>>>>> a J2EE container, and then have container integration as an >>>>>>>>>>> option. >>>>>>>>>>> >>>>>>>>>>> Configuration files are scattered everywhere. Anyone who has >>>>>>>>>>> deployed >>>>>>>>>>> OFBiz in a production environment will agree this is a problem. >>>>>>>>>>> Try >>>>>>>>>>> changing the HTTP/HTTPS and port settings - it is a nightmare. >>>>>>>>>>> Some >>>>>>>>>>> configuration settings are in nonsensical places. >>>>>>>>>>> >>>>>>>>>>> An abysmal lack of unit testing. I don't have an exact figure for >>>>>>>>>>> code >>>>>>>>>>> coverage, but my gut feeling is coverage is less than 10%. >>>>>>>>>>> Basically, >>>>>>>>>>> we >>>>>>>>>>> all have our fingers crossed - hoping that the framework code >>>>>>>>>>> works >>>>>>>>>>> as >>>>>>>>>>> expected. This was made painfully obvious a while back when I was >>>>>>>>>>> looking >>>>>>>>>>> at some entity caching code and thought to myself "this code can't >>>>>>>>>>> work." >>>>>>>>>>> So I wrote some entity cache unit tests and confirmed that the >>>>>>>>>>> entity >>>>>>>>>>> cache >>>>>>>>>>> had serious problems. Think about that - years passed with no >>>>>>>>>>> entity >>>>>>>>>>> cache >>>>>>>>>>> unit tests and consequently we had no idea it wasn't working. >>>>>>>>>>> >>>>>>>>>>> Fix Versus Rewrite >>>>>>>>>>> ------------------ >>>>>>>>>>> >>>>>>>>>>> Jira issues could be created for these problems and teams of >>>>>>>>>>> developers >>>>>>>>>>> could work to fix them. >>>>>>>>>>> >>>>>>>>>>> Or, we could create a branch and start over from scratch. This >>>>>>>>>>> time >>>>>>>>>>> around, there should be less push-back from people concerned about >>>>>>>>>>> backwards compatibility. A rewrite offers the advantage of >>>>>>>>>>> reconsidering >>>>>>>>>>> everything - like API design, general problem solving, and new >>>>>>>>>>> features. >>>>>>>>>>> >>>>>>>>>>> I created a Wiki page for a framework design: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> https://cwiki.apache.org/confluence/display/OFBADMIN/Another+Framework+Vision >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> but there hasn't been much interest in it. If the community >>>>>>>>>>> decides >>>>>>>>>>> to go >>>>>>>>>>> ahead with a rewrite, then please feel free to use the Wiki pages >>>>>>>>>>> as a >>>>>>>>>>> guide. >>>>>>>>>>> >>>>>>>>>>> Sandglass Foundation >>>>>>>>>>> -------------------- >>>>>>>>>>> >>>>>>>>>>> Like David, I came to the conclusion that a framework rewrite >>>>>>>>>>> would >>>>>>>>>>> be >>>>>>>>>>> easier outside the OFBiz community. So, I created my own library >>>>>>>>>>> called >>>>>>>>>>> Foundation: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> http://www.sandglass-software.com/products/sandglass/documents/Foundation_Brochure.pdf >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> (PDF) >>>>>>>>>>> >>>>>>>>>>> and I only mention it here to stress how wonderful it can be to >>>>>>>>>>> start >>>>>>>>>>> with >>>>>>>>>>> a clean slate and design an API that is concise yet powerful. >>>>>>>>>>> (Please >>>>>>>>>>> do >>>>>>>>>>> not discuss Foundation here, contact me privately if you want more >>>>>>>>>>> information.) >>>>>>>>>>> >>>>>>>>>>> Some examples of what can be done with a rewrite: >>>>>>>>>>> >>>>>>>>>>> A single configuration file >>>>>>>>>>> Use ANSI/ISO SQL SELECT statement strings instead of >>>>>>>>>>> constructing >>>>>>>>>>> complicated Java structures >>>>>>>>>>> Simultaneous asynchronous queries >>>>>>>>>>> Relational integrity across multiple datasources >>>>>>>>>>> Multi-table SELECT across multiple datasources >>>>>>>>>>> Automatic and transparent row version control >>>>>>>>>>> Automatic and transparent multi-language datasource support >>>>>>>>>>> Abstract entities (similar to SQL user types) >>>>>>>>>>> Service engine throttling (protects against server >>>>>>>>>>> over-utilization) >>>>>>>>>>> Simplified security (authorization) ( >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> https://cwiki.apache.org/confluence/display/OFBIZ/OFBiz+Security+Redesign >>>>>>>>>>> ) >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Pure interface-based API - so developers are free to modify >>>>>>>>>>> framework >>>>>>>>>>> behavior by using decorators >>>>>>>>>>> Thorough unit tests >>>>>>>>>>> >>>>>>>>>>> Benefits of a rewrite: >>>>>>>>>>> >>>>>>>>>>> Reduced resource requirements (lower hosting fees) >>>>>>>>>>> Reduce application development time - due to a simplified >>>>>>>>>>> API >>>>>>>>>>> Easier framework code maintenance >>>>>>>>>>> Better reliability >>>>>>>>>>> >>>>>>>>>>> Conclusion >>>>>>>>>>> ---------- >>>>>>>>>>> >>>>>>>>>>> Like I said at the start, this is all I will say about the >>>>>>>>>>> subject. >>>>>>>>>>> I'm >>>>>>>>>>> done trying to convince everyone. I hope someone agrees with me >>>>>>>>>>> and >>>>>>>>>>> they >>>>>>>>>>> are able to build support for the idea. >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Adrian Crum >>>>>>>>>>> Sandglass Software >>>>>>>>>>> www.sandglass-software.com >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> |
|
Hi Everyone,
Thank you for your contributions to this topic and the valuable input from everyone. May I suggest forming a team who is interested in taking the sub-project route given the (more or less) tendency of the community to move forward in that direction? Summarizing what I learned from this thread so far I think we should: - Form a team - Start a rewrite subproject - reintegrate to the framework at some point through a vote or a consensus process. So, any brave volunteers who would like to change the world a bit? Taher Alkhateeb ----- Original Message ----- From: "Jacques Le Roux" <[hidden email]> To: [hidden email] Sent: Friday, 16 October, 2015 1:06:17 PM Subject: Re: Why A Framework Rewrite Is Necessary I have not much time to comment at the moment, but at large this seems wise to me (in other words I kinda agree) Jacques Le 16/10/2015 02:48, Scott Gray a écrit : > I don't think this community is large enough to attempt to rewrite all of > our applications to suit Moqui while at the same time still supporting them > under the existing framework. That may well be the case for a homegrown > framework rewrite as well depending on how the applications would be > affected. This is the main reason why I tend to prefer incremental > improvements to the existing framework. But if people within the community > wanted to try for either of the "new framework" options, I certainly don't > have a problem with it and am happy to see some infrastructure donated to > the cause. > > There's simply no need to form a consensus on any major decision in the > early stages, proponents just need to be willing to accept that their > endeavors might not become part of the project. David certainly didn't > wait until he had a community backing him to go ahead and start work on > Moqui. The best way to get something done is to get started on it, if no > one follows then it's either a bad idea or you're doing a bad job of > selling it. > > By the way, I'm surprised I haven't seen any efforts within the Moqui > community to rewrite any of the OFBiz apps. I don't have any ideas as to > why that hasn't happened, but I'm curious about it. Surely it would be > easier for the Moqui community who already know the framework and are > probably also familiar with OFBiz to try this than us. > > Regards > Scott > > > On 16 October 2015 at 12:20, Hans Bakker <[hidden email]> wrote: > >> Well said: +1 >> >> >> On 15/10/15 22:13, Al Byers wrote: >> >>> Fair enough, Adrian, but is this a widely held position on OFBiz that as a >>> group you could produce a product that is so much better than Moqui that >>> it >>> would be worth the time and risk (the risk of burning cycles that you will >>> never get back only to have the venture never succeed.) Have all the >>> people >>> who would like to do a rewrite done the same gap analysis on Moqui vs what >>> you would like to see OFBiz be? Does everyone understand and agree that >>> the >>> deficiencies that you see in Moqui are important? My concern is that the >>> many people who understandably want a rewrite of OFBiz will be swayed by >>> your arguments without there being due diligence being done. I think that >>> there are a lot of veterans of OFBiz who have confidence in David's >>> development skills, but who are not pressing for a move to Moqui or for a >>> rewrite because they have so much invested in the current state of OFBiz >>> and recognized what a long shot a rewrite would be and how much work it >>> would require of them to move over. But anybody who is open to a rewrite >>> would be foolish not to consider the work done by the principal architect >>> of OFBiz; work that was meant to address deficiencies in OFBiz. >>> >>> On Thu, Oct 15, 2015 at 7:58 AM, Adrian Crum < >>> [hidden email]> wrote: >>> >>> Keep in mind that much of David's code in OFBiz has been rewritten. So >>>> yes, we CAN do a better job than him. Also keep in mind that Moqui >>>> duplicates some of the problems I listed - so by using Moqui, we keep the >>>> same problems instead of fixing them. On the other hand, it is David's >>>> responsibility to fix them, not ours. >>>> >>>> If we created a sub-project, then we would have the opportunity to review >>>> committer permissions and perhaps restrict access to the new code. >>>> >>>> Adrian Crum >>>> Sandglass Software >>>> www.sandglass-software.com >>>> >>>> On 10/15/2015 6:34 AM, Al Byers wrote: >>>> >>>> I was waiting for someone to bring this up. David Jones created Moqui >>>>> with >>>>> the same end in mind that the rewrite is meant to accomplish. Do you >>>>> think >>>>> that you will do a better job than him? Even if you could, would it be >>>>> so >>>>> much better that it warrants the effort that it would take? >>>>> >>>>> Is this a political thing? I don't know that David would give up control >>>>> of >>>>> Moqui Core and I, for one, would not want him to. So many of OFBiz's >>>>> problems are the result of ineptitude on the part of committers who did >>>>> not >>>>> know what they were doing (like me.) But I don't know that the Core will >>>>> change all that much going forward and there should be places in the >>>>> Mantle >>>>> to contribute and, most certainly, in the Crust. >>>>> >>>>> I know that there were some valid questions about licensing and project >>>>> management, but I would think that they could be worked out. >>>>> >>>>> On Thu, Oct 15, 2015 at 4:46 AM, Hans Bakker < >>>>> [hidden email]> >>>>> wrote: >>>>> >>>>> Why not skip this step, using moqui which is already up and running and >>>>> >>>>>> then as Adrian states as a next step: applications could be pulled down >>>>>> and >>>>>> adapted to it. >>>>>> >>>>>> Hans >>>>>> >>>>>> >>>>>> >>>>>> On 15/10/15 16:51, Jacques Le Roux wrote: >>>>>> >>>>>> I'm in the same mood than Paul and Scott. So a sub-project could indeed >>>>>> >>>>>>> be a solution. >>>>>>> >>>>>>> Jacques >>>>>>> >>>>>>> Le 15/10/2015 03:11, Adrian Crum a écrit : >>>>>>> >>>>>>> I agree that a sub-project would be nice to have. Once the new >>>>>>> framework >>>>>>> >>>>>>>> is up and running, applications could be pulled down and adapted to >>>>>>>> it. >>>>>>>> >>>>>>>> Adrian Crum >>>>>>>> Sandglass Software >>>>>>>> www.sandglass-software.com >>>>>>>> >>>>>>>> On 10/14/2015 5:53 PM, Ron Wheeler wrote: >>>>>>>> >>>>>>>> This seems to be very good advice. >>>>>>>> >>>>>>>>> A completely separate sub-project under OFBiz with its own mailing >>>>>>>>> lists >>>>>>>>> would keep the people together yet all allow the new framework the >>>>>>>>> flexibility to move forward. >>>>>>>>> >>>>>>>>> Ron >>>>>>>>> On 14/10/2015 8:27 PM, Scott Gray wrote: >>>>>>>>> >>>>>>>>> My advice is as follows: >>>>>>>>> >>>>>>>>>> 1. If people are interested, then get them together and start >>>>>>>>>> working >>>>>>>>>> on it. >>>>>>>>>> 2. Find somewhere to do the work. I don't think a branch is >>>>>>>>>> appropriate >>>>>>>>>> because it's completely new development rather than a >>>>>>>>>> refactoring. I >>>>>>>>>> don't >>>>>>>>>> have any objections to it being done under the ASF OFBiz umbrella >>>>>>>>>> (although >>>>>>>>>> I don't really see the need either). >>>>>>>>>> 3. Set up a separate mailing list for discussion. Generally I'd >>>>>>>>>> try >>>>>>>>>> and >>>>>>>>>> keep quiet about it in the early stages on the dev/user lists or >>>>>>>>>> other >>>>>>>>>> marketing channels because it could potentially harm adoption of >>>>>>>>>> our >>>>>>>>>> existing framework (think AngularJS 2.0). >>>>>>>>>> >>>>>>>>>> There really isn't any need to get early stage sign-off from the >>>>>>>>>> PMC >>>>>>>>>> or >>>>>>>>>> anyone else in the community. You only need enough PMC approval to >>>>>>>>>> get the >>>>>>>>>> required infrastructure sorted, which I don't think would be an >>>>>>>>>> issue. >>>>>>>>>> >From there, it's really up to the community to decide whether or >>>>>>>>>> not >>>>>>>>>> the >>>>>>>>>> thing will fly. >>>>>>>>>> >>>>>>>>>> Regards >>>>>>>>>> Scott >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 15 October 2015 at 08:21, Adrian Crum >>>>>>>>>> <[hidden email] >>>>>>>>>> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> I understand that Sharan brought up the framework rewrite subject >>>>>>>>>>> at >>>>>>>>>>> ApacheCon, and some attendees felt that the framework is fine and >>>>>>>>>>> no >>>>>>>>>>> action >>>>>>>>>>> needs to be taken. >>>>>>>>>>> >>>>>>>>>>> In this message, I will try to give a detailed explanation of why >>>>>>>>>>> a >>>>>>>>>>> framework rewrite is necessary. I don't plan to take any further >>>>>>>>>>> action on >>>>>>>>>>> this subject, because I've brought it up before without success, >>>>>>>>>>> and >>>>>>>>>>> I'm >>>>>>>>>>> tired of discussing it. It is my hope that the light bulb will >>>>>>>>>>> click >>>>>>>>>>> on in >>>>>>>>>>> someone's head and they will take action. >>>>>>>>>>> >>>>>>>>>>> My Background >>>>>>>>>>> ------------- >>>>>>>>>>> >>>>>>>>>>> I became a member of the OFBiz community in 2004. I immediately >>>>>>>>>>> started >>>>>>>>>>> making contributions to the project by supplying patches to the >>>>>>>>>>> issue >>>>>>>>>>> tracker. In 2007, I became a committer. Most of my initial work >>>>>>>>>>> was >>>>>>>>>>> on the >>>>>>>>>>> UI and some work in the applications (mainly Asset Maintenance and >>>>>>>>>>> Work >>>>>>>>>>> Effort). I stayed away from touching the framework code because it >>>>>>>>>>> was >>>>>>>>>>> deep, dark, and scary. >>>>>>>>>>> >>>>>>>>>>> Eventually, I started to understand how the framework code works >>>>>>>>>>> and I >>>>>>>>>>> made some minor modifications. As my understanding grew, I >>>>>>>>>>> progressed >>>>>>>>>>> to >>>>>>>>>>> rewriting large swaths of framework code - making it thread-safe, >>>>>>>>>>> fault >>>>>>>>>>> tolerant, efficient, and easier to use. >>>>>>>>>>> >>>>>>>>>>> I will list some of my contributions here, so everyone can have a >>>>>>>>>>> clear >>>>>>>>>>> understanding of my experience with the framework code: >>>>>>>>>>> >>>>>>>>>>> New Features >>>>>>>>>>> >>>>>>>>>>> User Preferences >>>>>>>>>>> >>>>>>>>>>> Visual Themes >>>>>>>>>>> >>>>>>>>>>> Custom UI Label XML File Format >>>>>>>>>>> >>>>>>>>>>> Temporal Expressions >>>>>>>>>>> >>>>>>>>>>> Data Type Conversion Framework >>>>>>>>>>> >>>>>>>>>>> Screen Widget Boundary Comments >>>>>>>>>>> >>>>>>>>>>> Metrics >>>>>>>>>>> >>>>>>>>>>> Integrations >>>>>>>>>>> >>>>>>>>>>> UEL >>>>>>>>>>> >>>>>>>>>>> iCalendar >>>>>>>>>>> >>>>>>>>>>> JSR 223 >>>>>>>>>>> >>>>>>>>>>> WebDAV >>>>>>>>>>> >>>>>>>>>>> LDAP >>>>>>>>>>> >>>>>>>>>>> Refactorings/Improvements >>>>>>>>>>> >>>>>>>>>>> FlexibleStringExpander >>>>>>>>>>> >>>>>>>>>>> FlexibleMapExpander >>>>>>>>>>> >>>>>>>>>>> FOP Integration >>>>>>>>>>> >>>>>>>>>>> FreeMarkerWorker >>>>>>>>>>> >>>>>>>>>>> Date-Time Handling >>>>>>>>>>> >>>>>>>>>>> Mini-language >>>>>>>>>>> >>>>>>>>>>> Job Scheduler >>>>>>>>>>> >>>>>>>>>>> In addition, I have performed innumerable framework bug fixes. >>>>>>>>>>> >>>>>>>>>>> So, the contents of this message come from years of experience >>>>>>>>>>> mucking >>>>>>>>>>> about in the framework code. >>>>>>>>>>> >>>>>>>>>>> Okay, let's get started... >>>>>>>>>>> >>>>>>>>>>> Initial Problem Statement >>>>>>>>>>> ------------------------- >>>>>>>>>>> >>>>>>>>>>> In 2009, David Jones started a framework rewrite in a branch: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> https://svn.apache.org/repos/asf/ofbiz/branches/executioncontext20090716 >>>>>>>>>>> >>>>>>>>>>> At the time, there was some agreement that a rewrite was >>>>>>>>>>> necessary, >>>>>>>>>>> but >>>>>>>>>>> there was disagreement as to how the rewrite should be >>>>>>>>>>> incorporated >>>>>>>>>>> into >>>>>>>>>>> the project: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> https://mail-archives.apache.org/mod_mbox/ofbiz-dev/200908.mbox/%3C455601.62605.qm@...%3E >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> There were concerns that a rewrite would break backward >>>>>>>>>>> compatibility. >>>>>>>>>>> Work on the rewrite branch stopped. Eventually, Jacopo suggested >>>>>>>>>>> the >>>>>>>>>>> community be more accepting of backward-incompatible changes: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> https://mail-archives.apache.org/mod_mbox/ofbiz-dev/201004.mbox/%3cD24F129D-4F9F-444E-84AF-ACA46F49990C@...%3e >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Despite an effort to convince David to proceed with the framework >>>>>>>>>>> rewrite, >>>>>>>>>>> he ended up doing it in a separate project: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> http://mail-archives.apache.org/mod_mbox/ofbiz-dev/201104.mbox/%3C07565C88-4023-4D24-93A3-A4906E86F939@...%3E >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> This page describes differences between OFBiz and Moqui, and >>>>>>>>>>> within >>>>>>>>>>> it you >>>>>>>>>>> can extract information on the problems David was trying to solve: >>>>>>>>>>> >>>>>>>>>>> http://sourceforge.net/p/moqui/discussion/1086127/thread/4c52f240/ >>>>>>>>>>> >>>>>>>>>>> There was an email he sent out on the OFBiz dev list where he >>>>>>>>>>> listed >>>>>>>>>>> the >>>>>>>>>>> problems he saw in the framework, but I can't find it. The rest of >>>>>>>>>>> this >>>>>>>>>>> message will include the issues he mentioned (the ones I >>>>>>>>>>> remember). >>>>>>>>>>> I >>>>>>>>>>> was >>>>>>>>>>> in agreement with him at the time, and I still agree that a >>>>>>>>>>> framework >>>>>>>>>>> rewrite is necessary. >>>>>>>>>>> >>>>>>>>>>> The Problems >>>>>>>>>>> ------------ >>>>>>>>>>> >>>>>>>>>>> Code is scattered everywhere - due to an initial effort to make >>>>>>>>>>> the >>>>>>>>>>> framework modular. This causes serious problems. The mere fact >>>>>>>>>>> that >>>>>>>>>>> components like entityext and securityext EXIST makes it clear >>>>>>>>>>> that >>>>>>>>>>> there >>>>>>>>>>> are problems - those components should not be there. Also, we run >>>>>>>>>>> into the >>>>>>>>>>> recurring problem of circular dependencies (component A will not >>>>>>>>>>> build >>>>>>>>>>> unless component B is built, and component B will not build unless >>>>>>>>>>> component A is built). >>>>>>>>>>> >>>>>>>>>>> Bad separation of concerns. There are far too many examples of >>>>>>>>>>> classes >>>>>>>>>>> that try to be everything to everyone. This makes debugging >>>>>>>>>>> difficult, and >>>>>>>>>>> it makes maintenance/improvements a nightmare. [Using an analogy, >>>>>>>>>>> consider >>>>>>>>>>> an automobile design where a spark plug is not separate from a >>>>>>>>>>> transmission. Instead, the automobile uses a >>>>>>>>>>> spark-plug-transmission. >>>>>>>>>>> So >>>>>>>>>>> when the engine is running rough because the spark plug is bad, >>>>>>>>>>> you >>>>>>>>>>> have to >>>>>>>>>>> replace the spark plug AND the transmission.] A good framework >>>>>>>>>>> example can >>>>>>>>>>> be found in my rewrite of the mini-language code. Originally, the >>>>>>>>>>> models >>>>>>>>>>> AND the script execution context both contained script behaviors - >>>>>>>>>>> making >>>>>>>>>>> debugging/improvements difficult. I changed it so only the models >>>>>>>>>>> contain >>>>>>>>>>> script behavior and the script execution context contains only the >>>>>>>>>>> script >>>>>>>>>>> execution state. >>>>>>>>>>> >>>>>>>>>>> Lack of good OO design. There are many places where a bit of >>>>>>>>>>> framework >>>>>>>>>>> functionality is contained in a single method that is hundreds or >>>>>>>>>>> thousands >>>>>>>>>>> of lines long. There is a term for that: Brittle Code. Code isn't >>>>>>>>>>> reused. >>>>>>>>>>> Instead, it is copy-and-pasted all over - so when a problem is >>>>>>>>>>> found >>>>>>>>>>> in the >>>>>>>>>>> C&P code, it has to be fixed in many places instead of one. >>>>>>>>>>> >>>>>>>>>>> Fail-slow design. There are a lot of places in low-level code >>>>>>>>>>> where >>>>>>>>>>> an >>>>>>>>>>> error condition is encountered, but instead of throwing an >>>>>>>>>>> exception, >>>>>>>>>>> the >>>>>>>>>>> error is ignored and maybe it is logged, or the code tries to >>>>>>>>>>> "guess" >>>>>>>>>>> at a >>>>>>>>>>> solution and then provide an arbitrary default behavior. I've seen >>>>>>>>>>> many >>>>>>>>>>> developers struggle with debugging a problem because they didn't >>>>>>>>>>> look >>>>>>>>>>> at >>>>>>>>>>> the logs, or because the error was not logged and there is no way >>>>>>>>>>> of >>>>>>>>>>> knowing what caused it. They end up spending hours single-stepping >>>>>>>>>>> through >>>>>>>>>>> code until it reaches the error. >>>>>>>>>>> >>>>>>>>>>> Out-of-date code. A good example is the use of Javolution. That >>>>>>>>>>> library >>>>>>>>>>> was beneficial in the Java 1.4 days, but it is not necessary today >>>>>>>>>>> because >>>>>>>>>>> of improved garbage collection. Another good example is DCL code. >>>>>>>>>>> DCL >>>>>>>>>>> was >>>>>>>>>>> used extensively in OFBiz, but it is clearly documented to be an >>>>>>>>>>> unreliable >>>>>>>>>>> design (I can get it to fail 90% of the time). Some DCL code has >>>>>>>>>>> been >>>>>>>>>>> replaced, but a lot of it is still there. >>>>>>>>>>> >>>>>>>>>>> Portions of the API are overly complicated. Some methods require a >>>>>>>>>>> collection of user-specified artifacts/arguments, which makes >>>>>>>>>>> client >>>>>>>>>>> code >>>>>>>>>>> complicated and verbose. (David solved that problem with his >>>>>>>>>>> Execution >>>>>>>>>>> Context.) Portions of the API are cluttered with unnecessary >>>>>>>>>>> "convenience >>>>>>>>>>> methods" - making the API harder to learn and memorize. In some >>>>>>>>>>> places, a >>>>>>>>>>> domain-specific API is spread across instance methods and static >>>>>>>>>>> methods >>>>>>>>>>> and across different classes - making the API hard to understand >>>>>>>>>>> and >>>>>>>>>>> use. >>>>>>>>>>> Yes, there can be good designs that require something like that, >>>>>>>>>>> but >>>>>>>>>>> in the >>>>>>>>>>> OFBiz framework, it exists because of a bad design, not a good >>>>>>>>>>> one. >>>>>>>>>>> >>>>>>>>>>> Use of thread-local variables. This makes multi-threaded design >>>>>>>>>>> impossible. The J2EE specification and the Servlet API require one >>>>>>>>>>> thread >>>>>>>>>>> per request (and most J2EE libraries depend on that behavior), so >>>>>>>>>>> the >>>>>>>>>>> current design makes sense from a J2EE perspective, but what if I >>>>>>>>>>> don't >>>>>>>>>>> want to run the framework in a J2EE container? Which leads to the >>>>>>>>>>> next >>>>>>>>>>> problem... >>>>>>>>>>> >>>>>>>>>>> Dependence on J2EE designs/APIs/libraries. There are developers in >>>>>>>>>>> the >>>>>>>>>>> Java community (myself included) who are beginning to question if >>>>>>>>>>> J2EE is >>>>>>>>>>> really necessary to run web applications. The folks at Atomikos >>>>>>>>>>> are >>>>>>>>>>> a >>>>>>>>>>> good >>>>>>>>>>> example. OFBiz does not use EJBs, so tying the framework to J2EE >>>>>>>>>>> does >>>>>>>>>>> not >>>>>>>>>>> make sense. It would be better if the framework was designed to >>>>>>>>>>> run >>>>>>>>>>> outside >>>>>>>>>>> a J2EE container, and then have container integration as an >>>>>>>>>>> option. >>>>>>>>>>> >>>>>>>>>>> Configuration files are scattered everywhere. Anyone who has >>>>>>>>>>> deployed >>>>>>>>>>> OFBiz in a production environment will agree this is a problem. >>>>>>>>>>> Try >>>>>>>>>>> changing the HTTP/HTTPS and port settings - it is a nightmare. >>>>>>>>>>> Some >>>>>>>>>>> configuration settings are in nonsensical places. >>>>>>>>>>> >>>>>>>>>>> An abysmal lack of unit testing. I don't have an exact figure for >>>>>>>>>>> code >>>>>>>>>>> coverage, but my gut feeling is coverage is less than 10%. >>>>>>>>>>> Basically, >>>>>>>>>>> we >>>>>>>>>>> all have our fingers crossed - hoping that the framework code >>>>>>>>>>> works >>>>>>>>>>> as >>>>>>>>>>> expected. This was made painfully obvious a while back when I was >>>>>>>>>>> looking >>>>>>>>>>> at some entity caching code and thought to myself "this code can't >>>>>>>>>>> work." >>>>>>>>>>> So I wrote some entity cache unit tests and confirmed that the >>>>>>>>>>> entity >>>>>>>>>>> cache >>>>>>>>>>> had serious problems. Think about that - years passed with no >>>>>>>>>>> entity >>>>>>>>>>> cache >>>>>>>>>>> unit tests and consequently we had no idea it wasn't working. >>>>>>>>>>> >>>>>>>>>>> Fix Versus Rewrite >>>>>>>>>>> ------------------ >>>>>>>>>>> >>>>>>>>>>> Jira issues could be created for these problems and teams of >>>>>>>>>>> developers >>>>>>>>>>> could work to fix them. >>>>>>>>>>> >>>>>>>>>>> Or, we could create a branch and start over from scratch. This >>>>>>>>>>> time >>>>>>>>>>> around, there should be less push-back from people concerned about >>>>>>>>>>> backwards compatibility. A rewrite offers the advantage of >>>>>>>>>>> reconsidering >>>>>>>>>>> everything - like API design, general problem solving, and new >>>>>>>>>>> features. >>>>>>>>>>> >>>>>>>>>>> I created a Wiki page for a framework design: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> https://cwiki.apache.org/confluence/display/OFBADMIN/Another+Framework+Vision >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> but there hasn't been much interest in it. If the community >>>>>>>>>>> decides >>>>>>>>>>> to go >>>>>>>>>>> ahead with a rewrite, then please feel free to use the Wiki pages >>>>>>>>>>> as a >>>>>>>>>>> guide. >>>>>>>>>>> >>>>>>>>>>> Sandglass Foundation >>>>>>>>>>> -------------------- >>>>>>>>>>> >>>>>>>>>>> Like David, I came to the conclusion that a framework rewrite >>>>>>>>>>> would >>>>>>>>>>> be >>>>>>>>>>> easier outside the OFBiz community. So, I created my own library >>>>>>>>>>> called >>>>>>>>>>> Foundation: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> http://www.sandglass-software.com/products/sandglass/documents/Foundation_Brochure.pdf >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> (PDF) >>>>>>>>>>> >>>>>>>>>>> and I only mention it here to stress how wonderful it can be to >>>>>>>>>>> start >>>>>>>>>>> with >>>>>>>>>>> a clean slate and design an API that is concise yet powerful. >>>>>>>>>>> (Please >>>>>>>>>>> do >>>>>>>>>>> not discuss Foundation here, contact me privately if you want more >>>>>>>>>>> information.) >>>>>>>>>>> >>>>>>>>>>> Some examples of what can be done with a rewrite: >>>>>>>>>>> >>>>>>>>>>> A single configuration file >>>>>>>>>>> Use ANSI/ISO SQL SELECT statement strings instead of >>>>>>>>>>> constructing >>>>>>>>>>> complicated Java structures >>>>>>>>>>> Simultaneous asynchronous queries >>>>>>>>>>> Relational integrity across multiple datasources >>>>>>>>>>> Multi-table SELECT across multiple datasources >>>>>>>>>>> Automatic and transparent row version control >>>>>>>>>>> Automatic and transparent multi-language datasource support >>>>>>>>>>> Abstract entities (similar to SQL user types) >>>>>>>>>>> Service engine throttling (protects against server >>>>>>>>>>> over-utilization) >>>>>>>>>>> Simplified security (authorization) ( >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> https://cwiki.apache.org/confluence/display/OFBIZ/OFBiz+Security+Redesign >>>>>>>>>>> ) >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Pure interface-based API - so developers are free to modify >>>>>>>>>>> framework >>>>>>>>>>> behavior by using decorators >>>>>>>>>>> Thorough unit tests >>>>>>>>>>> >>>>>>>>>>> Benefits of a rewrite: >>>>>>>>>>> >>>>>>>>>>> Reduced resource requirements (lower hosting fees) >>>>>>>>>>> Reduce application development time - due to a simplified >>>>>>>>>>> API >>>>>>>>>>> Easier framework code maintenance >>>>>>>>>>> Better reliability >>>>>>>>>>> >>>>>>>>>>> Conclusion >>>>>>>>>>> ---------- >>>>>>>>>>> >>>>>>>>>>> Like I said at the start, this is all I will say about the >>>>>>>>>>> subject. >>>>>>>>>>> I'm >>>>>>>>>>> done trying to convince everyone. I hope someone agrees with me >>>>>>>>>>> and >>>>>>>>>>> they >>>>>>>>>>> are able to build support for the idea. >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Adrian Crum >>>>>>>>>>> Sandglass Software >>>>>>>>>>> www.sandglass-software.com >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> |
Re: Why A Framework Rewrite Is Necessary
|
Administrator
|
In reply to this post by Pierre Smits
By and large, that also sounds wise to me. Like we said in French : "Il ne faut pas mettre la charrue avant les boeufs" (the cart before the horse)
But this also does not mean that nothing should be done. Simply we are not in a hurry and should take the time to collect, not the opinions (we have enough), but the ways to go. And we should not depend on projects to grow (easy to say...) I personally think Sharan is doing an excellent work a that, a kinda PM for the OFBiz project. We don't need a benevolent dictator (no misunderstandingDavid, I don't say that to you, I know you rejected the term and you always proved your openness and understanding, for those - if any - who would doubt, OFBiz is here to prove it) but we need a *more coherent team*. That's exactly how I feel Sharan's work, and it's not an easy one... Jacques Le 16/10/2015 09:41, Pierre Smits a écrit : > On Fri, Oct 16, 2015 at 5:31 AM, David E. Jones <[hidden email]> wrote: > >> >>> On 15 Oct 2015, at 07:58, Adrian Crum < >> [hidden email]> wrote: >>> Keep in mind that much of David's code in OFBiz has been rewritten. So >> yes, we CAN do a better job than him. >> >> I think there’s a name for this logical fallacy… >> >> And this could also be called a logical fallacy. But let us not make it > into a pissing contest again, like we had in the past regarding the > opposing viewpoints on this. > > >>> Also keep in mind that Moqui duplicates some of the problems I listed - >> so by using Moqui, we keep the same problems instead of fixing them. >> >> Could you be more specific, other than the type conversion stuff you >> mentioned many years ago (which I fully disagree with)? >> >> This is not about who is right or wrong, but where the community wants to > go. > > I understand the reluctance of the community, because the impact will be > huge. When looking at the data in OpenHub I see OFBiz having an estimate > effort spend of 519 person years vs 6 for the combined > Moqui-Mantle-HiveMindPM-PopCommerce suite. And one of the reasons behind it > is simple: Many more have worked on OFBiz (from day 1) than on the Moqui > suite. One could even argue that both directions took the same number of > years in duration to get where they are now. Without all the experiences > regarding the OFBiz product there couldn't have been an evolution called > the Moqui suite. > > Coming back to opting for a new direction, as Scott has stated we can have > this in a separate code repository (subproject, like many other Apache > projects do their work) even combined with a new JIRA an Wiki under the > umbrella of the OFBiz project. Based on the comments provided, this seems > like a logical choice to ensure that adopters of the current solution will > keep the support of the community while at the same time ensuring > containment of the new approach. > > But these are mere technical, supportive aspects. The more important issue > is what this new solution will encompass. There are talks of a rewrite, > which sounds like reinvention of the wheel. But I guess it is not like > that. Yet, taking decisions based on a few one-pagers (e.g. > http://www.sandglass-software.com/products/sandglass/documents/Foundation_Brochure.pdf) > are seldom done. Maybe it works for a single person, but I doubt it will > make a community fly. > > Whether fix or rewrite, choices will be made regarding the supporting 3rd > party libraries/solutions and the community needs to understand the impact > to get behind it. So before we embark on the coding trip, let us get into > trying to define a bit more what the new solution will encompass and get > consensus on that. > > Another issue regarding getting the community behind behind this new effort > is this: 'restrict access to the new code'. I guess this meant restrict > write access. Though understandable from a avoidance of dilution/scope > creep point of view, I see this as a wrong direction. This 'proposed' > exclusion of contributors of the kind will bring us back to where this > project came from: discrimination and favouritism. I even doubt that this > is possible under the current principles of the ASF. > Given that this is an enormous undertaking, we need to get as many on board > as possible. Not only to ensure that the decisions (at each level) will get > consensus, but also the ensure that every aspect down the line will get > addressed (e.g. documentation, test definitions, marketing/promotions, etc) > in order to get this kite flying. > > Best regards, > > Pierre Smits > > *OFBiz Extensions Marketplace* > http://oem.ofbizci.net/oci-2/ > |
Re: Why A Framework Rewrite Is Necessary
|
|
In reply to this post by Pierre Smits
Before we start to decide who can access code, we need to decide on a
roadmap for the new framework. How different will the API be from the current framework in each of the areas that the framework will offer services? How modular will it be? Foundation identifies - Core with 3 main areas of functionality. Can they be separated into separate projects with clean interfaces? are there more projects such as authentication, persistence, logging and audit (see below) that are shared across the Foundation Core high level features. - Script - Imex - Connect - seems to a number of projects here that could be tackled separately. Is this it? Will there be an application isolation layer that will support OFBiz's current interaction with the Framework. This should also be a separate project where OFBiz knowledge is really valuable. What will go underneath the covers? Spring-Boot , Spring JPA, Hibernate, etc. How many containers will be supported. Tomcat, Jetty, Glassfish, Spring-Boot. How many persistence options will be supported? SQL, No-SQL How many authentication services will be supported - internal, LDAP, Oauth, Google, LinkedIn, Facebook. What administration functions will be offered? How? JConsole, REST, browser/mobile apps. How delivered? Installer, Docker image, VM image, What demo apps? What test framework(s)? Test Applications. What would be a reasonable set of functionality to be released in version 1.0? Minimum useful framework. How many people would it take to do this in a reasonable timeframe? Ron On 16/10/2015 3:41 AM, Pierre Smits wrote: > On Fri, Oct 16, 2015 at 5:31 AM, David E. Jones <[hidden email]> wrote: > >> >>> On 15 Oct 2015, at 07:58, Adrian Crum < >> [hidden email]> wrote: >>> Keep in mind that much of David's code in OFBiz has been rewritten. So >> yes, we CAN do a better job than him. >> >> I think there’s a name for this logical fallacy… >> >> And this could also be called a logical fallacy. But let us not make it > into a pissing contest again, like we had in the past regarding the > opposing viewpoints on this. > > >>> Also keep in mind that Moqui duplicates some of the problems I listed - >> so by using Moqui, we keep the same problems instead of fixing them. >> >> Could you be more specific, other than the type conversion stuff you >> mentioned many years ago (which I fully disagree with)? >> >> This is not about who is right or wrong, but where the community wants to > go. > > I understand the reluctance of the community, because the impact will be > huge. When looking at the data in OpenHub I see OFBiz having an estimate > effort spend of 519 person years vs 6 for the combined > Moqui-Mantle-HiveMindPM-PopCommerce suite. And one of the reasons behind it > is simple: Many more have worked on OFBiz (from day 1) than on the Moqui > suite. One could even argue that both directions took the same number of > years in duration to get where they are now. Without all the experiences > regarding the OFBiz product there couldn't have been an evolution called > the Moqui suite. > > Coming back to opting for a new direction, as Scott has stated we can have > this in a separate code repository (subproject, like many other Apache > projects do their work) even combined with a new JIRA an Wiki under the > umbrella of the OFBiz project. Based on the comments provided, this seems > like a logical choice to ensure that adopters of the current solution will > keep the support of the community while at the same time ensuring > containment of the new approach. > > But these are mere technical, supportive aspects. The more important issue > is what this new solution will encompass. There are talks of a rewrite, > which sounds like reinvention of the wheel. But I guess it is not like > that. Yet, taking decisions based on a few one-pagers (e.g. > http://www.sandglass-software.com/products/sandglass/documents/Foundation_Brochure.pdf) > are seldom done. Maybe it works for a single person, but I doubt it will > make a community fly. > > Whether fix or rewrite, choices will be made regarding the supporting 3rd > party libraries/solutions and the community needs to understand the impact > to get behind it. So before we embark on the coding trip, let us get into > trying to define a bit more what the new solution will encompass and get > consensus on that. > > Another issue regarding getting the community behind behind this new effort > is this: 'restrict access to the new code'. I guess this meant restrict > write access. Though understandable from a avoidance of dilution/scope > creep point of view, I see this as a wrong direction. This 'proposed' > exclusion of contributors of the kind will bring us back to where this > project came from: discrimination and favouritism. I even doubt that this > is possible under the current principles of the ASF. > Given that this is an enormous undertaking, we need to get as many on board > as possible. Not only to ensure that the decisions (at each level) will get > consensus, but also the ensure that every aspect down the line will get > addressed (e.g. documentation, test definitions, marketing/promotions, etc) > in order to get this kite flying. > > Best regards, > > Pierre Smits > > *OFBiz Extensions Marketplace* > http://oem.ofbizci.net/oci-2/ > -- Ron Wheeler President Artifact Software Inc email: [hidden email] skype: ronaldmwheeler phone: 866-970-2435, ext 102 |
Re: Why A Framework Rewrite Is Necessary
|
|
Ron,
Please stop referring to Foundation. I only referenced that library to make a point about how much easier it is come up with better designs when you start with a clean slate. Otherwise, it has no bearing on this discussion. Adrian Crum Sandglass Software www.sandglass-software.com On 10/16/2015 7:47 AM, Ron Wheeler wrote: > Before we start to decide who can access code, we need to decide on a > roadmap for the new framework. > How different will the API be from the current framework in each of the > areas that the framework will offer services? > > How modular will it be? > Foundation identifies > - Core with 3 main areas of functionality. Can they be separated into > separate projects with clean interfaces? are there more projects such as > authentication, persistence, logging and audit (see below) that are > shared across the Foundation Core high level features. > - Script > - Imex > - Connect - seems to a number of projects here that could be tackled > separately. > Is this it? > > Will there be an application isolation layer that will support OFBiz's > current interaction with the Framework. This should also be a separate > project where OFBiz knowledge is really valuable. > > What will go underneath the covers? Spring-Boot , Spring JPA, Hibernate, > etc. > > How many containers will be supported. Tomcat, Jetty, Glassfish, > Spring-Boot. > > How many persistence options will be supported? SQL, No-SQL > > How many authentication services will be supported - internal, LDAP, > Oauth, Google, LinkedIn, Facebook. > > What administration functions will be offered? How? JConsole, REST, > browser/mobile apps. > > How delivered? Installer, Docker image, VM image, > > What demo apps? > > What test framework(s)? Test Applications. > > What would be a reasonable set of functionality to be released in > version 1.0? Minimum useful framework. > > How many people would it take to do this in a reasonable timeframe? > > Ron > > > On 16/10/2015 3:41 AM, Pierre Smits wrote: >> On Fri, Oct 16, 2015 at 5:31 AM, David E. Jones <[hidden email]> wrote: >> >>> >>>> On 15 Oct 2015, at 07:58, Adrian Crum < >>> [hidden email]> wrote: >>>> Keep in mind that much of David's code in OFBiz has been rewritten. So >>> yes, we CAN do a better job than him. >>> >>> I think there’s a name for this logical fallacy… >>> >>> And this could also be called a logical fallacy. But let us not make it >> into a pissing contest again, like we had in the past regarding the >> opposing viewpoints on this. >> >> >>>> Also keep in mind that Moqui duplicates some of the problems I listed - >>> so by using Moqui, we keep the same problems instead of fixing them. >>> >>> Could you be more specific, other than the type conversion stuff you >>> mentioned many years ago (which I fully disagree with)? >>> >>> This is not about who is right or wrong, but where the community >>> wants to >> go. >> >> I understand the reluctance of the community, because the impact will be >> huge. When looking at the data in OpenHub I see OFBiz having an estimate >> effort spend of 519 person years vs 6 for the combined >> Moqui-Mantle-HiveMindPM-PopCommerce suite. And one of the reasons >> behind it >> is simple: Many more have worked on OFBiz (from day 1) than on the Moqui >> suite. One could even argue that both directions took the same number of >> years in duration to get where they are now. Without all the experiences >> regarding the OFBiz product there couldn't have been an evolution called >> the Moqui suite. >> >> Coming back to opting for a new direction, as Scott has stated we can >> have >> this in a separate code repository (subproject, like many other Apache >> projects do their work) even combined with a new JIRA an Wiki under the >> umbrella of the OFBiz project. Based on the comments provided, this seems >> like a logical choice to ensure that adopters of the current solution >> will >> keep the support of the community while at the same time ensuring >> containment of the new approach. >> >> But these are mere technical, supportive aspects. The more important >> issue >> is what this new solution will encompass. There are talks of a rewrite, >> which sounds like reinvention of the wheel. But I guess it is not like >> that. Yet, taking decisions based on a few one-pagers (e.g. >> http://www.sandglass-software.com/products/sandglass/documents/Foundation_Brochure.pdf) >> >> are seldom done. Maybe it works for a single person, but I doubt it will >> make a community fly. >> >> Whether fix or rewrite, choices will be made regarding the supporting 3rd >> party libraries/solutions and the community needs to understand the >> impact >> to get behind it. So before we embark on the coding trip, let us get into >> trying to define a bit more what the new solution will encompass and get >> consensus on that. >> >> Another issue regarding getting the community behind behind this new >> effort >> is this: 'restrict access to the new code'. I guess this meant restrict >> write access. Though understandable from a avoidance of dilution/scope >> creep point of view, I see this as a wrong direction. This 'proposed' >> exclusion of contributors of the kind will bring us back to where this >> project came from: discrimination and favouritism. I even doubt that this >> is possible under the current principles of the ASF. >> Given that this is an enormous undertaking, we need to get as many on >> board >> as possible. Not only to ensure that the decisions (at each level) >> will get >> consensus, but also the ensure that every aspect down the line will get >> addressed (e.g. documentation, test definitions, marketing/promotions, >> etc) >> in order to get this kite flying. >> >> Best regards, >> >> Pierre Smits >> >> *OFBiz Extensions Marketplace* >> http://oem.ofbizci.net/oci-2/ >> > > |
Re: Why A Framework Rewrite Is Necessary
|
|
In reply to this post by David E. Jones-2
Is there any enthusiasm for incorporating Moqui as a separate third
party dependency into the ERP? What is involved? Does anyone have any sort of plan for using Moqui as a drop-in replacement for the current Framework? A blog article would be a good start in adding some real flesh to this idea if it makes any sense. If there is a desire for a new OFBiz framework, we should start by adding some details in the wiki. I can not edit this or add comments. Once we see how far we can get in a roadmap and functional overview, we can start to look at sub-projects. It is too early now as far as I can tell. Ron On 15/10/2015 11:31 PM, David E. Jones wrote: > >> On 15 Oct 2015, at 07:58, Adrian Crum <[hidden email]> wrote: >> >> Keep in mind that much of David's code in OFBiz has been rewritten. So yes, we CAN do a better job than him. > I think there’s a name for this logical fallacy… > >> Also keep in mind that Moqui duplicates some of the problems I listed - so by using Moqui, we keep the same problems instead of fixing them. > Could you be more specific, other than the type conversion stuff you mentioned many years ago (which I fully disagree with)? > > -David > > >> On the other hand, it is David's responsibility to fix them, not ours. >> >> If we created a sub-project, then we would have the opportunity to review committer permissions and perhaps restrict access to the new code. >> >> Adrian Crum >> Sandglass Software >> www.sandglass-software.com >> >> On 10/15/2015 6:34 AM, Al Byers wrote: >>> I was waiting for someone to bring this up. David Jones created Moqui with >>> the same end in mind that the rewrite is meant to accomplish. Do you think >>> that you will do a better job than him? Even if you could, would it be so >>> much better that it warrants the effort that it would take? >>> >>> Is this a political thing? I don't know that David would give up control of >>> Moqui Core and I, for one, would not want him to. So many of OFBiz's >>> problems are the result of ineptitude on the part of committers who did not >>> know what they were doing (like me.) But I don't know that the Core will >>> change all that much going forward and there should be places in the Mantle >>> to contribute and, most certainly, in the Crust. >>> >>> I know that there were some valid questions about licensing and project >>> management, but I would think that they could be worked out. >>> >>> On Thu, Oct 15, 2015 at 4:46 AM, Hans Bakker <[hidden email]> >>> wrote: >>> >>>> Why not skip this step, using moqui which is already up and running and >>>> then as Adrian states as a next step: applications could be pulled down and >>>> adapted to it. >>>> >>>> Hans >>>> >>>> >>>> >>>> On 15/10/15 16:51, Jacques Le Roux wrote: >>>> >>>>> I'm in the same mood than Paul and Scott. So a sub-project could indeed >>>>> be a solution. >>>>> >>>>> Jacques >>>>> >>>>> Le 15/10/2015 03:11, Adrian Crum a écrit : >>>>> >>>>>> I agree that a sub-project would be nice to have. Once the new framework >>>>>> is up and running, applications could be pulled down and adapted to it. >>>>>> >>>>>> Adrian Crum >>>>>> Sandglass Software >>>>>> www.sandglass-software.com >>>>>> >>>>>> On 10/14/2015 5:53 PM, Ron Wheeler wrote: >>>>>> >>>>>>> This seems to be very good advice. >>>>>>> A completely separate sub-project under OFBiz with its own mailing lists >>>>>>> would keep the people together yet all allow the new framework the >>>>>>> flexibility to move forward. >>>>>>> >>>>>>> Ron >>>>>>> On 14/10/2015 8:27 PM, Scott Gray wrote: >>>>>>> >>>>>>>> My advice is as follows: >>>>>>>> 1. If people are interested, then get them together and start working >>>>>>>> on it. >>>>>>>> 2. Find somewhere to do the work. I don't think a branch is >>>>>>>> appropriate >>>>>>>> because it's completely new development rather than a refactoring. I >>>>>>>> don't >>>>>>>> have any objections to it being done under the ASF OFBiz umbrella >>>>>>>> (although >>>>>>>> I don't really see the need either). >>>>>>>> 3. Set up a separate mailing list for discussion. Generally I'd try >>>>>>>> and >>>>>>>> keep quiet about it in the early stages on the dev/user lists or other >>>>>>>> marketing channels because it could potentially harm adoption of our >>>>>>>> existing framework (think AngularJS 2.0). >>>>>>>> >>>>>>>> There really isn't any need to get early stage sign-off from the PMC or >>>>>>>> anyone else in the community. You only need enough PMC approval to >>>>>>>> get the >>>>>>>> required infrastructure sorted, which I don't think would be an issue. >>>>>>>> >From there, it's really up to the community to decide whether or not >>>>>>>> the >>>>>>>> thing will fly. >>>>>>>> >>>>>>>> Regards >>>>>>>> Scott >>>>>>>> >>>>>>>> >>>>>>>> On 15 October 2015 at 08:21, Adrian Crum >>>>>>>> <[hidden email] >>>>>>>> >>>>>>>>> wrote: >>>>>>>>> I understand that Sharan brought up the framework rewrite subject at >>>>>>>>> ApacheCon, and some attendees felt that the framework is fine and no >>>>>>>>> action >>>>>>>>> needs to be taken. >>>>>>>>> >>>>>>>>> In this message, I will try to give a detailed explanation of why a >>>>>>>>> framework rewrite is necessary. I don't plan to take any further >>>>>>>>> action on >>>>>>>>> this subject, because I've brought it up before without success, and >>>>>>>>> I'm >>>>>>>>> tired of discussing it. It is my hope that the light bulb will click >>>>>>>>> on in >>>>>>>>> someone's head and they will take action. >>>>>>>>> >>>>>>>>> My Background >>>>>>>>> ------------- >>>>>>>>> >>>>>>>>> I became a member of the OFBiz community in 2004. I immediately >>>>>>>>> started >>>>>>>>> making contributions to the project by supplying patches to the issue >>>>>>>>> tracker. In 2007, I became a committer. Most of my initial work was >>>>>>>>> on the >>>>>>>>> UI and some work in the applications (mainly Asset Maintenance and >>>>>>>>> Work >>>>>>>>> Effort). I stayed away from touching the framework code because it was >>>>>>>>> deep, dark, and scary. >>>>>>>>> >>>>>>>>> Eventually, I started to understand how the framework code works and I >>>>>>>>> made some minor modifications. As my understanding grew, I progressed >>>>>>>>> to >>>>>>>>> rewriting large swaths of framework code - making it thread-safe, >>>>>>>>> fault >>>>>>>>> tolerant, efficient, and easier to use. >>>>>>>>> >>>>>>>>> I will list some of my contributions here, so everyone can have a >>>>>>>>> clear >>>>>>>>> understanding of my experience with the framework code: >>>>>>>>> >>>>>>>>> New Features >>>>>>>>> >>>>>>>>> User Preferences >>>>>>>>> >>>>>>>>> Visual Themes >>>>>>>>> >>>>>>>>> Custom UI Label XML File Format >>>>>>>>> >>>>>>>>> Temporal Expressions >>>>>>>>> >>>>>>>>> Data Type Conversion Framework >>>>>>>>> >>>>>>>>> Screen Widget Boundary Comments >>>>>>>>> >>>>>>>>> Metrics >>>>>>>>> >>>>>>>>> Integrations >>>>>>>>> >>>>>>>>> UEL >>>>>>>>> >>>>>>>>> iCalendar >>>>>>>>> >>>>>>>>> JSR 223 >>>>>>>>> >>>>>>>>> WebDAV >>>>>>>>> >>>>>>>>> LDAP >>>>>>>>> >>>>>>>>> Refactorings/Improvements >>>>>>>>> >>>>>>>>> FlexibleStringExpander >>>>>>>>> >>>>>>>>> FlexibleMapExpander >>>>>>>>> >>>>>>>>> FOP Integration >>>>>>>>> >>>>>>>>> FreeMarkerWorker >>>>>>>>> >>>>>>>>> Date-Time Handling >>>>>>>>> >>>>>>>>> Mini-language >>>>>>>>> >>>>>>>>> Job Scheduler >>>>>>>>> >>>>>>>>> In addition, I have performed innumerable framework bug fixes. >>>>>>>>> >>>>>>>>> So, the contents of this message come from years of experience mucking >>>>>>>>> about in the framework code. >>>>>>>>> >>>>>>>>> Okay, let's get started... >>>>>>>>> >>>>>>>>> Initial Problem Statement >>>>>>>>> ------------------------- >>>>>>>>> >>>>>>>>> In 2009, David Jones started a framework rewrite in a branch: >>>>>>>>> >>>>>>>>> >>>>>>>>> https://svn.apache.org/repos/asf/ofbiz/branches/executioncontext20090716 >>>>>>>>> >>>>>>>>> At the time, there was some agreement that a rewrite was necessary, >>>>>>>>> but >>>>>>>>> there was disagreement as to how the rewrite should be incorporated >>>>>>>>> into >>>>>>>>> the project: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> https://mail-archives.apache.org/mod_mbox/ofbiz-dev/200908.mbox/%3C455601.62605.qm@...%3E >>>>>>>>> >>>>>>>>> >>>>>>>>> There were concerns that a rewrite would break backward compatibility. >>>>>>>>> Work on the rewrite branch stopped. Eventually, Jacopo suggested the >>>>>>>>> community be more accepting of backward-incompatible changes: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> https://mail-archives.apache.org/mod_mbox/ofbiz-dev/201004.mbox/%3cD24F129D-4F9F-444E-84AF-ACA46F49990C@...%3e >>>>>>>>> >>>>>>>>> >>>>>>>>> Despite an effort to convince David to proceed with the framework >>>>>>>>> rewrite, >>>>>>>>> he ended up doing it in a separate project: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> http://mail-archives.apache.org/mod_mbox/ofbiz-dev/201104.mbox/%3C07565C88-4023-4D24-93A3-A4906E86F939@...%3E >>>>>>>>> >>>>>>>>> >>>>>>>>> This page describes differences between OFBiz and Moqui, and within >>>>>>>>> it you >>>>>>>>> can extract information on the problems David was trying to solve: >>>>>>>>> >>>>>>>>> http://sourceforge.net/p/moqui/discussion/1086127/thread/4c52f240/ >>>>>>>>> >>>>>>>>> There was an email he sent out on the OFBiz dev list where he listed >>>>>>>>> the >>>>>>>>> problems he saw in the framework, but I can't find it. The rest of >>>>>>>>> this >>>>>>>>> message will include the issues he mentioned (the ones I remember). I >>>>>>>>> was >>>>>>>>> in agreement with him at the time, and I still agree that a framework >>>>>>>>> rewrite is necessary. >>>>>>>>> >>>>>>>>> The Problems >>>>>>>>> ------------ >>>>>>>>> >>>>>>>>> Code is scattered everywhere - due to an initial effort to make the >>>>>>>>> framework modular. This causes serious problems. The mere fact that >>>>>>>>> components like entityext and securityext EXIST makes it clear that >>>>>>>>> there >>>>>>>>> are problems - those components should not be there. Also, we run >>>>>>>>> into the >>>>>>>>> recurring problem of circular dependencies (component A will not build >>>>>>>>> unless component B is built, and component B will not build unless >>>>>>>>> component A is built). >>>>>>>>> >>>>>>>>> Bad separation of concerns. There are far too many examples of classes >>>>>>>>> that try to be everything to everyone. This makes debugging >>>>>>>>> difficult, and >>>>>>>>> it makes maintenance/improvements a nightmare. [Using an analogy, >>>>>>>>> consider >>>>>>>>> an automobile design where a spark plug is not separate from a >>>>>>>>> transmission. Instead, the automobile uses a spark-plug-transmission. >>>>>>>>> So >>>>>>>>> when the engine is running rough because the spark plug is bad, you >>>>>>>>> have to >>>>>>>>> replace the spark plug AND the transmission.] A good framework >>>>>>>>> example can >>>>>>>>> be found in my rewrite of the mini-language code. Originally, the >>>>>>>>> models >>>>>>>>> AND the script execution context both contained script behaviors - >>>>>>>>> making >>>>>>>>> debugging/improvements difficult. I changed it so only the models >>>>>>>>> contain >>>>>>>>> script behavior and the script execution context contains only the >>>>>>>>> script >>>>>>>>> execution state. >>>>>>>>> >>>>>>>>> Lack of good OO design. There are many places where a bit of framework >>>>>>>>> functionality is contained in a single method that is hundreds or >>>>>>>>> thousands >>>>>>>>> of lines long. There is a term for that: Brittle Code. Code isn't >>>>>>>>> reused. >>>>>>>>> Instead, it is copy-and-pasted all over - so when a problem is found >>>>>>>>> in the >>>>>>>>> C&P code, it has to be fixed in many places instead of one. >>>>>>>>> >>>>>>>>> Fail-slow design. There are a lot of places in low-level code where an >>>>>>>>> error condition is encountered, but instead of throwing an exception, >>>>>>>>> the >>>>>>>>> error is ignored and maybe it is logged, or the code tries to "guess" >>>>>>>>> at a >>>>>>>>> solution and then provide an arbitrary default behavior. I've seen >>>>>>>>> many >>>>>>>>> developers struggle with debugging a problem because they didn't look >>>>>>>>> at >>>>>>>>> the logs, or because the error was not logged and there is no way of >>>>>>>>> knowing what caused it. They end up spending hours single-stepping >>>>>>>>> through >>>>>>>>> code until it reaches the error. >>>>>>>>> >>>>>>>>> Out-of-date code. A good example is the use of Javolution. That >>>>>>>>> library >>>>>>>>> was beneficial in the Java 1.4 days, but it is not necessary today >>>>>>>>> because >>>>>>>>> of improved garbage collection. Another good example is DCL code. DCL >>>>>>>>> was >>>>>>>>> used extensively in OFBiz, but it is clearly documented to be an >>>>>>>>> unreliable >>>>>>>>> design (I can get it to fail 90% of the time). Some DCL code has been >>>>>>>>> replaced, but a lot of it is still there. >>>>>>>>> >>>>>>>>> Portions of the API are overly complicated. Some methods require a >>>>>>>>> collection of user-specified artifacts/arguments, which makes client >>>>>>>>> code >>>>>>>>> complicated and verbose. (David solved that problem with his Execution >>>>>>>>> Context.) Portions of the API are cluttered with unnecessary >>>>>>>>> "convenience >>>>>>>>> methods" - making the API harder to learn and memorize. In some >>>>>>>>> places, a >>>>>>>>> domain-specific API is spread across instance methods and static >>>>>>>>> methods >>>>>>>>> and across different classes - making the API hard to understand and >>>>>>>>> use. >>>>>>>>> Yes, there can be good designs that require something like that, but >>>>>>>>> in the >>>>>>>>> OFBiz framework, it exists because of a bad design, not a good one. >>>>>>>>> >>>>>>>>> Use of thread-local variables. This makes multi-threaded design >>>>>>>>> impossible. The J2EE specification and the Servlet API require one >>>>>>>>> thread >>>>>>>>> per request (and most J2EE libraries depend on that behavior), so the >>>>>>>>> current design makes sense from a J2EE perspective, but what if I >>>>>>>>> don't >>>>>>>>> want to run the framework in a J2EE container? Which leads to the next >>>>>>>>> problem... >>>>>>>>> >>>>>>>>> Dependence on J2EE designs/APIs/libraries. There are developers in the >>>>>>>>> Java community (myself included) who are beginning to question if >>>>>>>>> J2EE is >>>>>>>>> really necessary to run web applications. The folks at Atomikos are a >>>>>>>>> good >>>>>>>>> example. OFBiz does not use EJBs, so tying the framework to J2EE does >>>>>>>>> not >>>>>>>>> make sense. It would be better if the framework was designed to run >>>>>>>>> outside >>>>>>>>> a J2EE container, and then have container integration as an option. >>>>>>>>> >>>>>>>>> Configuration files are scattered everywhere. Anyone who has deployed >>>>>>>>> OFBiz in a production environment will agree this is a problem. Try >>>>>>>>> changing the HTTP/HTTPS and port settings - it is a nightmare. Some >>>>>>>>> configuration settings are in nonsensical places. >>>>>>>>> >>>>>>>>> An abysmal lack of unit testing. I don't have an exact figure for code >>>>>>>>> coverage, but my gut feeling is coverage is less than 10%. Basically, >>>>>>>>> we >>>>>>>>> all have our fingers crossed - hoping that the framework code works as >>>>>>>>> expected. This was made painfully obvious a while back when I was >>>>>>>>> looking >>>>>>>>> at some entity caching code and thought to myself "this code can't >>>>>>>>> work." >>>>>>>>> So I wrote some entity cache unit tests and confirmed that the entity >>>>>>>>> cache >>>>>>>>> had serious problems. Think about that - years passed with no entity >>>>>>>>> cache >>>>>>>>> unit tests and consequently we had no idea it wasn't working. >>>>>>>>> >>>>>>>>> Fix Versus Rewrite >>>>>>>>> ------------------ >>>>>>>>> >>>>>>>>> Jira issues could be created for these problems and teams of >>>>>>>>> developers >>>>>>>>> could work to fix them. >>>>>>>>> >>>>>>>>> Or, we could create a branch and start over from scratch. This time >>>>>>>>> around, there should be less push-back from people concerned about >>>>>>>>> backwards compatibility. A rewrite offers the advantage of >>>>>>>>> reconsidering >>>>>>>>> everything - like API design, general problem solving, and new >>>>>>>>> features. >>>>>>>>> >>>>>>>>> I created a Wiki page for a framework design: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> https://cwiki.apache.org/confluence/display/OFBADMIN/Another+Framework+Vision >>>>>>>>> >>>>>>>>> >>>>>>>>> but there hasn't been much interest in it. If the community decides >>>>>>>>> to go >>>>>>>>> ahead with a rewrite, then please feel free to use the Wiki pages as a >>>>>>>>> guide. >>>>>>>>> >>>>>>>>> Sandglass Foundation >>>>>>>>> -------------------- >>>>>>>>> >>>>>>>>> Like David, I came to the conclusion that a framework rewrite would be >>>>>>>>> easier outside the OFBiz community. So, I created my own library >>>>>>>>> called >>>>>>>>> Foundation: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> http://www.sandglass-software.com/products/sandglass/documents/Foundation_Brochure.pdf >>>>>>>>> >>>>>>>>> >>>>>>>>> (PDF) >>>>>>>>> >>>>>>>>> and I only mention it here to stress how wonderful it can be to start >>>>>>>>> with >>>>>>>>> a clean slate and design an API that is concise yet powerful. (Please >>>>>>>>> do >>>>>>>>> not discuss Foundation here, contact me privately if you want more >>>>>>>>> information.) >>>>>>>>> >>>>>>>>> Some examples of what can be done with a rewrite: >>>>>>>>> >>>>>>>>> A single configuration file >>>>>>>>> Use ANSI/ISO SQL SELECT statement strings instead of constructing >>>>>>>>> complicated Java structures >>>>>>>>> Simultaneous asynchronous queries >>>>>>>>> Relational integrity across multiple datasources >>>>>>>>> Multi-table SELECT across multiple datasources >>>>>>>>> Automatic and transparent row version control >>>>>>>>> Automatic and transparent multi-language datasource support >>>>>>>>> Abstract entities (similar to SQL user types) >>>>>>>>> Service engine throttling (protects against server >>>>>>>>> over-utilization) >>>>>>>>> Simplified security (authorization) ( >>>>>>>>> >>>>>>>>> https://cwiki.apache.org/confluence/display/OFBIZ/OFBiz+Security+Redesign) >>>>>>>>> >>>>>>>>> >>>>>>>>> Pure interface-based API - so developers are free to modify >>>>>>>>> framework >>>>>>>>> behavior by using decorators >>>>>>>>> Thorough unit tests >>>>>>>>> >>>>>>>>> Benefits of a rewrite: >>>>>>>>> >>>>>>>>> Reduced resource requirements (lower hosting fees) >>>>>>>>> Reduce application development time - due to a simplified API >>>>>>>>> Easier framework code maintenance >>>>>>>>> Better reliability >>>>>>>>> >>>>>>>>> Conclusion >>>>>>>>> ---------- >>>>>>>>> >>>>>>>>> Like I said at the start, this is all I will say about the subject. >>>>>>>>> I'm >>>>>>>>> done trying to convince everyone. I hope someone agrees with me and >>>>>>>>> they >>>>>>>>> are able to build support for the idea. >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Adrian Crum >>>>>>>>> Sandglass Software >>>>>>>>> www.sandglass-software.com >>>>>>>>> >>>>>>>>> >>>>>>> > -- Ron Wheeler President Artifact Software Inc email: [hidden email] skype: ronaldmwheeler phone: 866-970-2435, ext 102 |
Re: Why A Framework Rewrite Is Necessary
|
|
That discussion occurred in April, and a vote was taken:
http://ofbiz.135035.n4.nabble.com/VOTE-Begin-Replacing-OFBiz-Framework-With-Moqui-td4667500.html Adrian Crum Sandglass Software www.sandglass-software.com On 10/16/2015 8:51 AM, Ron Wheeler wrote: > Is there any enthusiasm for incorporating Moqui as a separate third > party dependency into the ERP? > What is involved? Does anyone have any sort of plan for using Moqui as a > drop-in replacement for the current Framework? > A blog article would be a good start in adding some real flesh to this > idea if it makes any sense. > > If there is a desire for a new OFBiz framework, we should start by > adding some details in the wiki. > I can not edit this or add comments. > > Once we see how far we can get in a roadmap and functional overview, we > can start to look at sub-projects. > It is too early now as far as I can tell. > > Ron > > On 15/10/2015 11:31 PM, David E. Jones wrote: >> >>> On 15 Oct 2015, at 07:58, Adrian Crum >>> <[hidden email]> wrote: >>> >>> Keep in mind that much of David's code in OFBiz has been rewritten. >>> So yes, we CAN do a better job than him. >> I think there’s a name for this logical fallacy… >> >>> Also keep in mind that Moqui duplicates some of the problems I listed >>> - so by using Moqui, we keep the same problems instead of fixing them. >> Could you be more specific, other than the type conversion stuff you >> mentioned many years ago (which I fully disagree with)? >> >> -David >> >> >>> On the other hand, it is David's responsibility to fix them, not ours. >>> >>> If we created a sub-project, then we would have the opportunity to >>> review committer permissions and perhaps restrict access to the new >>> code. >>> >>> Adrian Crum >>> Sandglass Software >>> www.sandglass-software.com >>> >>> On 10/15/2015 6:34 AM, Al Byers wrote: >>>> I was waiting for someone to bring this up. David Jones created >>>> Moqui with >>>> the same end in mind that the rewrite is meant to accomplish. Do you >>>> think >>>> that you will do a better job than him? Even if you could, would it >>>> be so >>>> much better that it warrants the effort that it would take? >>>> >>>> Is this a political thing? I don't know that David would give up >>>> control of >>>> Moqui Core and I, for one, would not want him to. So many of OFBiz's >>>> problems are the result of ineptitude on the part of committers who >>>> did not >>>> know what they were doing (like me.) But I don't know that the Core >>>> will >>>> change all that much going forward and there should be places in the >>>> Mantle >>>> to contribute and, most certainly, in the Crust. >>>> >>>> I know that there were some valid questions about licensing and project >>>> management, but I would think that they could be worked out. >>>> >>>> On Thu, Oct 15, 2015 at 4:46 AM, Hans Bakker >>>> <[hidden email]> >>>> wrote: >>>> >>>>> Why not skip this step, using moqui which is already up and running >>>>> and >>>>> then as Adrian states as a next step: applications could be pulled >>>>> down and >>>>> adapted to it. >>>>> >>>>> Hans >>>>> >>>>> >>>>> >>>>> On 15/10/15 16:51, Jacques Le Roux wrote: >>>>> >>>>>> I'm in the same mood than Paul and Scott. So a sub-project could >>>>>> indeed >>>>>> be a solution. >>>>>> >>>>>> Jacques >>>>>> >>>>>> Le 15/10/2015 03:11, Adrian Crum a écrit : >>>>>> >>>>>>> I agree that a sub-project would be nice to have. Once the new >>>>>>> framework >>>>>>> is up and running, applications could be pulled down and adapted >>>>>>> to it. >>>>>>> >>>>>>> Adrian Crum >>>>>>> Sandglass Software >>>>>>> www.sandglass-software.com >>>>>>> >>>>>>> On 10/14/2015 5:53 PM, Ron Wheeler wrote: >>>>>>> >>>>>>>> This seems to be very good advice. >>>>>>>> A completely separate sub-project under OFBiz with its own >>>>>>>> mailing lists >>>>>>>> would keep the people together yet all allow the new framework the >>>>>>>> flexibility to move forward. >>>>>>>> >>>>>>>> Ron >>>>>>>> On 14/10/2015 8:27 PM, Scott Gray wrote: >>>>>>>> >>>>>>>>> My advice is as follows: >>>>>>>>> 1. If people are interested, then get them together and start >>>>>>>>> working >>>>>>>>> on it. >>>>>>>>> 2. Find somewhere to do the work. I don't think a branch is >>>>>>>>> appropriate >>>>>>>>> because it's completely new development rather than a >>>>>>>>> refactoring. I >>>>>>>>> don't >>>>>>>>> have any objections to it being done under the ASF OFBiz umbrella >>>>>>>>> (although >>>>>>>>> I don't really see the need either). >>>>>>>>> 3. Set up a separate mailing list for discussion. Generally >>>>>>>>> I'd try >>>>>>>>> and >>>>>>>>> keep quiet about it in the early stages on the dev/user lists >>>>>>>>> or other >>>>>>>>> marketing channels because it could potentially harm adoption >>>>>>>>> of our >>>>>>>>> existing framework (think AngularJS 2.0). >>>>>>>>> >>>>>>>>> There really isn't any need to get early stage sign-off from >>>>>>>>> the PMC or >>>>>>>>> anyone else in the community. You only need enough PMC >>>>>>>>> approval to >>>>>>>>> get the >>>>>>>>> required infrastructure sorted, which I don't think would be an >>>>>>>>> issue. >>>>>>>>> >From there, it's really up to the community to decide whether >>>>>>>>> or not >>>>>>>>> the >>>>>>>>> thing will fly. >>>>>>>>> >>>>>>>>> Regards >>>>>>>>> Scott >>>>>>>>> >>>>>>>>> >>>>>>>>> On 15 October 2015 at 08:21, Adrian Crum >>>>>>>>> <[hidden email] >>>>>>>>> >>>>>>>>>> wrote: >>>>>>>>>> I understand that Sharan brought up the framework rewrite >>>>>>>>>> subject at >>>>>>>>>> ApacheCon, and some attendees felt that the framework is fine >>>>>>>>>> and no >>>>>>>>>> action >>>>>>>>>> needs to be taken. >>>>>>>>>> >>>>>>>>>> In this message, I will try to give a detailed explanation of >>>>>>>>>> why a >>>>>>>>>> framework rewrite is necessary. I don't plan to take any further >>>>>>>>>> action on >>>>>>>>>> this subject, because I've brought it up before without >>>>>>>>>> success, and >>>>>>>>>> I'm >>>>>>>>>> tired of discussing it. It is my hope that the light bulb will >>>>>>>>>> click >>>>>>>>>> on in >>>>>>>>>> someone's head and they will take action. >>>>>>>>>> >>>>>>>>>> My Background >>>>>>>>>> ------------- >>>>>>>>>> >>>>>>>>>> I became a member of the OFBiz community in 2004. I immediately >>>>>>>>>> started >>>>>>>>>> making contributions to the project by supplying patches to >>>>>>>>>> the issue >>>>>>>>>> tracker. In 2007, I became a committer. Most of my initial >>>>>>>>>> work was >>>>>>>>>> on the >>>>>>>>>> UI and some work in the applications (mainly Asset Maintenance >>>>>>>>>> and >>>>>>>>>> Work >>>>>>>>>> Effort). I stayed away from touching the framework code >>>>>>>>>> because it was >>>>>>>>>> deep, dark, and scary. >>>>>>>>>> >>>>>>>>>> Eventually, I started to understand how the framework code >>>>>>>>>> works and I >>>>>>>>>> made some minor modifications. As my understanding grew, I >>>>>>>>>> progressed >>>>>>>>>> to >>>>>>>>>> rewriting large swaths of framework code - making it thread-safe, >>>>>>>>>> fault >>>>>>>>>> tolerant, efficient, and easier to use. >>>>>>>>>> >>>>>>>>>> I will list some of my contributions here, so everyone can have a >>>>>>>>>> clear >>>>>>>>>> understanding of my experience with the framework code: >>>>>>>>>> >>>>>>>>>> New Features >>>>>>>>>> >>>>>>>>>> User Preferences >>>>>>>>>> >>>>>>>>>> Visual Themes >>>>>>>>>> >>>>>>>>>> Custom UI Label XML File Format >>>>>>>>>> >>>>>>>>>> Temporal Expressions >>>>>>>>>> >>>>>>>>>> Data Type Conversion Framework >>>>>>>>>> >>>>>>>>>> Screen Widget Boundary Comments >>>>>>>>>> >>>>>>>>>> Metrics >>>>>>>>>> >>>>>>>>>> Integrations >>>>>>>>>> >>>>>>>>>> UEL >>>>>>>>>> >>>>>>>>>> iCalendar >>>>>>>>>> >>>>>>>>>> JSR 223 >>>>>>>>>> >>>>>>>>>> WebDAV >>>>>>>>>> >>>>>>>>>> LDAP >>>>>>>>>> >>>>>>>>>> Refactorings/Improvements >>>>>>>>>> >>>>>>>>>> FlexibleStringExpander >>>>>>>>>> >>>>>>>>>> FlexibleMapExpander >>>>>>>>>> >>>>>>>>>> FOP Integration >>>>>>>>>> >>>>>>>>>> FreeMarkerWorker >>>>>>>>>> >>>>>>>>>> Date-Time Handling >>>>>>>>>> >>>>>>>>>> Mini-language >>>>>>>>>> >>>>>>>>>> Job Scheduler >>>>>>>>>> >>>>>>>>>> In addition, I have performed innumerable framework bug fixes. >>>>>>>>>> >>>>>>>>>> So, the contents of this message come from years of experience >>>>>>>>>> mucking >>>>>>>>>> about in the framework code. >>>>>>>>>> >>>>>>>>>> Okay, let's get started... >>>>>>>>>> >>>>>>>>>> Initial Problem Statement >>>>>>>>>> ------------------------- >>>>>>>>>> >>>>>>>>>> In 2009, David Jones started a framework rewrite in a branch: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> https://svn.apache.org/repos/asf/ofbiz/branches/executioncontext20090716 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> At the time, there was some agreement that a rewrite was >>>>>>>>>> necessary, >>>>>>>>>> but >>>>>>>>>> there was disagreement as to how the rewrite should be >>>>>>>>>> incorporated >>>>>>>>>> into >>>>>>>>>> the project: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> https://mail-archives.apache.org/mod_mbox/ofbiz-dev/200908.mbox/%3C455601.62605.qm@...%3E >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> There were concerns that a rewrite would break backward >>>>>>>>>> compatibility. >>>>>>>>>> Work on the rewrite branch stopped. Eventually, Jacopo >>>>>>>>>> suggested the >>>>>>>>>> community be more accepting of backward-incompatible changes: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> https://mail-archives.apache.org/mod_mbox/ofbiz-dev/201004.mbox/%3cD24F129D-4F9F-444E-84AF-ACA46F49990C@...%3e >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Despite an effort to convince David to proceed with the framework >>>>>>>>>> rewrite, >>>>>>>>>> he ended up doing it in a separate project: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> http://mail-archives.apache.org/mod_mbox/ofbiz-dev/201104.mbox/%3C07565C88-4023-4D24-93A3-A4906E86F939@...%3E >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> This page describes differences between OFBiz and Moqui, and >>>>>>>>>> within >>>>>>>>>> it you >>>>>>>>>> can extract information on the problems David was trying to >>>>>>>>>> solve: >>>>>>>>>> >>>>>>>>>> http://sourceforge.net/p/moqui/discussion/1086127/thread/4c52f240/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> There was an email he sent out on the OFBiz dev list where he >>>>>>>>>> listed >>>>>>>>>> the >>>>>>>>>> problems he saw in the framework, but I can't find it. The >>>>>>>>>> rest of >>>>>>>>>> this >>>>>>>>>> message will include the issues he mentioned (the ones I >>>>>>>>>> remember). I >>>>>>>>>> was >>>>>>>>>> in agreement with him at the time, and I still agree that a >>>>>>>>>> framework >>>>>>>>>> rewrite is necessary. >>>>>>>>>> >>>>>>>>>> The Problems >>>>>>>>>> ------------ >>>>>>>>>> >>>>>>>>>> Code is scattered everywhere - due to an initial effort to >>>>>>>>>> make the >>>>>>>>>> framework modular. This causes serious problems. The mere fact >>>>>>>>>> that >>>>>>>>>> components like entityext and securityext EXIST makes it clear >>>>>>>>>> that >>>>>>>>>> there >>>>>>>>>> are problems - those components should not be there. Also, we run >>>>>>>>>> into the >>>>>>>>>> recurring problem of circular dependencies (component A will >>>>>>>>>> not build >>>>>>>>>> unless component B is built, and component B will not build >>>>>>>>>> unless >>>>>>>>>> component A is built). >>>>>>>>>> >>>>>>>>>> Bad separation of concerns. There are far too many examples of >>>>>>>>>> classes >>>>>>>>>> that try to be everything to everyone. This makes debugging >>>>>>>>>> difficult, and >>>>>>>>>> it makes maintenance/improvements a nightmare. [Using an analogy, >>>>>>>>>> consider >>>>>>>>>> an automobile design where a spark plug is not separate from a >>>>>>>>>> transmission. Instead, the automobile uses a >>>>>>>>>> spark-plug-transmission. >>>>>>>>>> So >>>>>>>>>> when the engine is running rough because the spark plug is >>>>>>>>>> bad, you >>>>>>>>>> have to >>>>>>>>>> replace the spark plug AND the transmission.] A good framework >>>>>>>>>> example can >>>>>>>>>> be found in my rewrite of the mini-language code. Originally, the >>>>>>>>>> models >>>>>>>>>> AND the script execution context both contained script >>>>>>>>>> behaviors - >>>>>>>>>> making >>>>>>>>>> debugging/improvements difficult. I changed it so only the models >>>>>>>>>> contain >>>>>>>>>> script behavior and the script execution context contains only >>>>>>>>>> the >>>>>>>>>> script >>>>>>>>>> execution state. >>>>>>>>>> >>>>>>>>>> Lack of good OO design. There are many places where a bit of >>>>>>>>>> framework >>>>>>>>>> functionality is contained in a single method that is hundreds or >>>>>>>>>> thousands >>>>>>>>>> of lines long. There is a term for that: Brittle Code. Code isn't >>>>>>>>>> reused. >>>>>>>>>> Instead, it is copy-and-pasted all over - so when a problem is >>>>>>>>>> found >>>>>>>>>> in the >>>>>>>>>> C&P code, it has to be fixed in many places instead of one. >>>>>>>>>> >>>>>>>>>> Fail-slow design. There are a lot of places in low-level code >>>>>>>>>> where an >>>>>>>>>> error condition is encountered, but instead of throwing an >>>>>>>>>> exception, >>>>>>>>>> the >>>>>>>>>> error is ignored and maybe it is logged, or the code tries to >>>>>>>>>> "guess" >>>>>>>>>> at a >>>>>>>>>> solution and then provide an arbitrary default behavior. I've >>>>>>>>>> seen >>>>>>>>>> many >>>>>>>>>> developers struggle with debugging a problem because they >>>>>>>>>> didn't look >>>>>>>>>> at >>>>>>>>>> the logs, or because the error was not logged and there is no >>>>>>>>>> way of >>>>>>>>>> knowing what caused it. They end up spending hours >>>>>>>>>> single-stepping >>>>>>>>>> through >>>>>>>>>> code until it reaches the error. >>>>>>>>>> >>>>>>>>>> Out-of-date code. A good example is the use of Javolution. That >>>>>>>>>> library >>>>>>>>>> was beneficial in the Java 1.4 days, but it is not necessary >>>>>>>>>> today >>>>>>>>>> because >>>>>>>>>> of improved garbage collection. Another good example is DCL >>>>>>>>>> code. DCL >>>>>>>>>> was >>>>>>>>>> used extensively in OFBiz, but it is clearly documented to be an >>>>>>>>>> unreliable >>>>>>>>>> design (I can get it to fail 90% of the time). Some DCL code >>>>>>>>>> has been >>>>>>>>>> replaced, but a lot of it is still there. >>>>>>>>>> >>>>>>>>>> Portions of the API are overly complicated. Some methods >>>>>>>>>> require a >>>>>>>>>> collection of user-specified artifacts/arguments, which makes >>>>>>>>>> client >>>>>>>>>> code >>>>>>>>>> complicated and verbose. (David solved that problem with his >>>>>>>>>> Execution >>>>>>>>>> Context.) Portions of the API are cluttered with unnecessary >>>>>>>>>> "convenience >>>>>>>>>> methods" - making the API harder to learn and memorize. In some >>>>>>>>>> places, a >>>>>>>>>> domain-specific API is spread across instance methods and static >>>>>>>>>> methods >>>>>>>>>> and across different classes - making the API hard to >>>>>>>>>> understand and >>>>>>>>>> use. >>>>>>>>>> Yes, there can be good designs that require something like >>>>>>>>>> that, but >>>>>>>>>> in the >>>>>>>>>> OFBiz framework, it exists because of a bad design, not a good >>>>>>>>>> one. >>>>>>>>>> >>>>>>>>>> Use of thread-local variables. This makes multi-threaded design >>>>>>>>>> impossible. The J2EE specification and the Servlet API require >>>>>>>>>> one >>>>>>>>>> thread >>>>>>>>>> per request (and most J2EE libraries depend on that behavior), >>>>>>>>>> so the >>>>>>>>>> current design makes sense from a J2EE perspective, but what if I >>>>>>>>>> don't >>>>>>>>>> want to run the framework in a J2EE container? Which leads to >>>>>>>>>> the next >>>>>>>>>> problem... >>>>>>>>>> >>>>>>>>>> Dependence on J2EE designs/APIs/libraries. There are >>>>>>>>>> developers in the >>>>>>>>>> Java community (myself included) who are beginning to question if >>>>>>>>>> J2EE is >>>>>>>>>> really necessary to run web applications. The folks at >>>>>>>>>> Atomikos are a >>>>>>>>>> good >>>>>>>>>> example. OFBiz does not use EJBs, so tying the framework to >>>>>>>>>> J2EE does >>>>>>>>>> not >>>>>>>>>> make sense. It would be better if the framework was designed >>>>>>>>>> to run >>>>>>>>>> outside >>>>>>>>>> a J2EE container, and then have container integration as an >>>>>>>>>> option. >>>>>>>>>> >>>>>>>>>> Configuration files are scattered everywhere. Anyone who has >>>>>>>>>> deployed >>>>>>>>>> OFBiz in a production environment will agree this is a >>>>>>>>>> problem. Try >>>>>>>>>> changing the HTTP/HTTPS and port settings - it is a nightmare. >>>>>>>>>> Some >>>>>>>>>> configuration settings are in nonsensical places. >>>>>>>>>> >>>>>>>>>> An abysmal lack of unit testing. I don't have an exact figure >>>>>>>>>> for code >>>>>>>>>> coverage, but my gut feeling is coverage is less than 10%. >>>>>>>>>> Basically, >>>>>>>>>> we >>>>>>>>>> all have our fingers crossed - hoping that the framework code >>>>>>>>>> works as >>>>>>>>>> expected. This was made painfully obvious a while back when I was >>>>>>>>>> looking >>>>>>>>>> at some entity caching code and thought to myself "this code >>>>>>>>>> can't >>>>>>>>>> work." >>>>>>>>>> So I wrote some entity cache unit tests and confirmed that the >>>>>>>>>> entity >>>>>>>>>> cache >>>>>>>>>> had serious problems. Think about that - years passed with no >>>>>>>>>> entity >>>>>>>>>> cache >>>>>>>>>> unit tests and consequently we had no idea it wasn't working. >>>>>>>>>> >>>>>>>>>> Fix Versus Rewrite >>>>>>>>>> ------------------ >>>>>>>>>> >>>>>>>>>> Jira issues could be created for these problems and teams of >>>>>>>>>> developers >>>>>>>>>> could work to fix them. >>>>>>>>>> >>>>>>>>>> Or, we could create a branch and start over from scratch. This >>>>>>>>>> time >>>>>>>>>> around, there should be less push-back from people concerned >>>>>>>>>> about >>>>>>>>>> backwards compatibility. A rewrite offers the advantage of >>>>>>>>>> reconsidering >>>>>>>>>> everything - like API design, general problem solving, and new >>>>>>>>>> features. >>>>>>>>>> >>>>>>>>>> I created a Wiki page for a framework design: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> https://cwiki.apache.org/confluence/display/OFBADMIN/Another+Framework+Vision >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> but there hasn't been much interest in it. If the community >>>>>>>>>> decides >>>>>>>>>> to go >>>>>>>>>> ahead with a rewrite, then please feel free to use the Wiki >>>>>>>>>> pages as a >>>>>>>>>> guide. >>>>>>>>>> >>>>>>>>>> Sandglass Foundation >>>>>>>>>> -------------------- >>>>>>>>>> >>>>>>>>>> Like David, I came to the conclusion that a framework rewrite >>>>>>>>>> would be >>>>>>>>>> easier outside the OFBiz community. So, I created my own library >>>>>>>>>> called >>>>>>>>>> Foundation: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> http://www.sandglass-software.com/products/sandglass/documents/Foundation_Brochure.pdf >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> (PDF) >>>>>>>>>> >>>>>>>>>> and I only mention it here to stress how wonderful it can be >>>>>>>>>> to start >>>>>>>>>> with >>>>>>>>>> a clean slate and design an API that is concise yet powerful. >>>>>>>>>> (Please >>>>>>>>>> do >>>>>>>>>> not discuss Foundation here, contact me privately if you want >>>>>>>>>> more >>>>>>>>>> information.) >>>>>>>>>> >>>>>>>>>> Some examples of what can be done with a rewrite: >>>>>>>>>> >>>>>>>>>> A single configuration file >>>>>>>>>> Use ANSI/ISO SQL SELECT statement strings instead of >>>>>>>>>> constructing >>>>>>>>>> complicated Java structures >>>>>>>>>> Simultaneous asynchronous queries >>>>>>>>>> Relational integrity across multiple datasources >>>>>>>>>> Multi-table SELECT across multiple datasources >>>>>>>>>> Automatic and transparent row version control >>>>>>>>>> Automatic and transparent multi-language datasource support >>>>>>>>>> Abstract entities (similar to SQL user types) >>>>>>>>>> Service engine throttling (protects against server >>>>>>>>>> over-utilization) >>>>>>>>>> Simplified security (authorization) ( >>>>>>>>>> >>>>>>>>>> https://cwiki.apache.org/confluence/display/OFBIZ/OFBiz+Security+Redesign) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Pure interface-based API - so developers are free to modify >>>>>>>>>> framework >>>>>>>>>> behavior by using decorators >>>>>>>>>> Thorough unit tests >>>>>>>>>> >>>>>>>>>> Benefits of a rewrite: >>>>>>>>>> >>>>>>>>>> Reduced resource requirements (lower hosting fees) >>>>>>>>>> Reduce application development time - due to a >>>>>>>>>> simplified API >>>>>>>>>> Easier framework code maintenance >>>>>>>>>> Better reliability >>>>>>>>>> >>>>>>>>>> Conclusion >>>>>>>>>> ---------- >>>>>>>>>> >>>>>>>>>> Like I said at the start, this is all I will say about the >>>>>>>>>> subject. >>>>>>>>>> I'm >>>>>>>>>> done trying to convince everyone. I hope someone agrees with >>>>>>>>>> me and >>>>>>>>>> they >>>>>>>>>> are able to build support for the idea. >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Adrian Crum >>>>>>>>>> Sandglass Software >>>>>>>>>> www.sandglass-software.com >>>>>>>>>> >>>>>>>>>> >>>>>>>> >> > > |
Re: Why A Framework Rewrite Is Necessary
|
|
Adrian,
That was a vote for replacing parts of the framework. Now we are discussing replacing the ofbiz framework as a whole by moqui as a jar. Leaving the framework control outside of ofbiz. At my company we envision the future of OFBiz using Moqui and we are prepared to invest a number of our employees making the move. We see no future in replacing the framework with something else especially a rewrite. Regards, Hans Bakker http://antwebsystems.com htttp://growerp.com On 17/10/15 01:54, Adrian Crum wrote: > That discussion occurred in April, and a vote was taken: > > http://ofbiz.135035.n4.nabble.com/VOTE-Begin-Replacing-OFBiz-Framework-With-Moqui-td4667500.html > > > Adrian Crum > Sandglass Software > www.sandglass-software.com > > On 10/16/2015 8:51 AM, Ron Wheeler wrote: >> Is there any enthusiasm for incorporating Moqui as a separate third >> party dependency into the ERP? >> What is involved? Does anyone have any sort of plan for using Moqui as a >> drop-in replacement for the current Framework? >> A blog article would be a good start in adding some real flesh to this >> idea if it makes any sense. >> >> If there is a desire for a new OFBiz framework, we should start by >> adding some details in the wiki. >> I can not edit this or add comments. >> >> Once we see how far we can get in a roadmap and functional overview, we >> can start to look at sub-projects. >> It is too early now as far as I can tell. >> >> Ron >> >> On 15/10/2015 11:31 PM, David E. Jones wrote: >>> >>>> On 15 Oct 2015, at 07:58, Adrian Crum >>>> <[hidden email]> wrote: >>>> >>>> Keep in mind that much of David's code in OFBiz has been rewritten. >>>> So yes, we CAN do a better job than him. >>> I think there’s a name for this logical fallacy… >>> >>>> Also keep in mind that Moqui duplicates some of the problems I listed >>>> - so by using Moqui, we keep the same problems instead of fixing them. >>> Could you be more specific, other than the type conversion stuff you >>> mentioned many years ago (which I fully disagree with)? >>> >>> -David >>> >>> >>>> On the other hand, it is David's responsibility to fix them, not ours. >>>> >>>> If we created a sub-project, then we would have the opportunity to >>>> review committer permissions and perhaps restrict access to the new >>>> code. >>>> >>>> Adrian Crum >>>> Sandglass Software >>>> www.sandglass-software.com >>>> >>>> On 10/15/2015 6:34 AM, Al Byers wrote: >>>>> I was waiting for someone to bring this up. David Jones created >>>>> Moqui with >>>>> the same end in mind that the rewrite is meant to accomplish. Do you >>>>> think >>>>> that you will do a better job than him? Even if you could, would it >>>>> be so >>>>> much better that it warrants the effort that it would take? >>>>> >>>>> Is this a political thing? I don't know that David would give up >>>>> control of >>>>> Moqui Core and I, for one, would not want him to. So many of OFBiz's >>>>> problems are the result of ineptitude on the part of committers who >>>>> did not >>>>> know what they were doing (like me.) But I don't know that the Core >>>>> will >>>>> change all that much going forward and there should be places in the >>>>> Mantle >>>>> to contribute and, most certainly, in the Crust. >>>>> >>>>> I know that there were some valid questions about licensing and >>>>> project >>>>> management, but I would think that they could be worked out. >>>>> >>>>> On Thu, Oct 15, 2015 at 4:46 AM, Hans Bakker >>>>> <[hidden email]> >>>>> wrote: >>>>> >>>>>> Why not skip this step, using moqui which is already up and running >>>>>> and >>>>>> then as Adrian states as a next step: applications could be pulled >>>>>> down and >>>>>> adapted to it. >>>>>> >>>>>> Hans >>>>>> >>>>>> >>>>>> >>>>>> On 15/10/15 16:51, Jacques Le Roux wrote: >>>>>> >>>>>>> I'm in the same mood than Paul and Scott. So a sub-project could >>>>>>> indeed >>>>>>> be a solution. >>>>>>> >>>>>>> Jacques >>>>>>> >>>>>>> Le 15/10/2015 03:11, Adrian Crum a écrit : >>>>>>> >>>>>>>> I agree that a sub-project would be nice to have. Once the new >>>>>>>> framework >>>>>>>> is up and running, applications could be pulled down and adapted >>>>>>>> to it. >>>>>>>> >>>>>>>> Adrian Crum >>>>>>>> Sandglass Software >>>>>>>> www.sandglass-software.com >>>>>>>> >>>>>>>> On 10/14/2015 5:53 PM, Ron Wheeler wrote: >>>>>>>> >>>>>>>>> This seems to be very good advice. >>>>>>>>> A completely separate sub-project under OFBiz with its own >>>>>>>>> mailing lists >>>>>>>>> would keep the people together yet all allow the new framework >>>>>>>>> the >>>>>>>>> flexibility to move forward. >>>>>>>>> >>>>>>>>> Ron >>>>>>>>> On 14/10/2015 8:27 PM, Scott Gray wrote: >>>>>>>>> >>>>>>>>>> My advice is as follows: >>>>>>>>>> 1. If people are interested, then get them together and start >>>>>>>>>> working >>>>>>>>>> on it. >>>>>>>>>> 2. Find somewhere to do the work. I don't think a branch is >>>>>>>>>> appropriate >>>>>>>>>> because it's completely new development rather than a >>>>>>>>>> refactoring. I >>>>>>>>>> don't >>>>>>>>>> have any objections to it being done under the ASF OFBiz >>>>>>>>>> umbrella >>>>>>>>>> (although >>>>>>>>>> I don't really see the need either). >>>>>>>>>> 3. Set up a separate mailing list for discussion. Generally >>>>>>>>>> I'd try >>>>>>>>>> and >>>>>>>>>> keep quiet about it in the early stages on the dev/user lists >>>>>>>>>> or other >>>>>>>>>> marketing channels because it could potentially harm adoption >>>>>>>>>> of our >>>>>>>>>> existing framework (think AngularJS 2.0). >>>>>>>>>> >>>>>>>>>> There really isn't any need to get early stage sign-off from >>>>>>>>>> the PMC or >>>>>>>>>> anyone else in the community. You only need enough PMC >>>>>>>>>> approval to >>>>>>>>>> get the >>>>>>>>>> required infrastructure sorted, which I don't think would be an >>>>>>>>>> issue. >>>>>>>>>> >From there, it's really up to the community to decide whether >>>>>>>>>> or not >>>>>>>>>> the >>>>>>>>>> thing will fly. >>>>>>>>>> >>>>>>>>>> Regards >>>>>>>>>> Scott >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 15 October 2015 at 08:21, Adrian Crum >>>>>>>>>> <[hidden email] >>>>>>>>>> >>>>>>>>>>> wrote: >>>>>>>>>>> I understand that Sharan brought up the framework rewrite >>>>>>>>>>> subject at >>>>>>>>>>> ApacheCon, and some attendees felt that the framework is fine >>>>>>>>>>> and no >>>>>>>>>>> action >>>>>>>>>>> needs to be taken. >>>>>>>>>>> >>>>>>>>>>> In this message, I will try to give a detailed explanation of >>>>>>>>>>> why a >>>>>>>>>>> framework rewrite is necessary. I don't plan to take any >>>>>>>>>>> further >>>>>>>>>>> action on >>>>>>>>>>> this subject, because I've brought it up before without >>>>>>>>>>> success, and >>>>>>>>>>> I'm >>>>>>>>>>> tired of discussing it. It is my hope that the light bulb will >>>>>>>>>>> click >>>>>>>>>>> on in >>>>>>>>>>> someone's head and they will take action. >>>>>>>>>>> >>>>>>>>>>> My Background >>>>>>>>>>> ------------- >>>>>>>>>>> >>>>>>>>>>> I became a member of the OFBiz community in 2004. I immediately >>>>>>>>>>> started >>>>>>>>>>> making contributions to the project by supplying patches to >>>>>>>>>>> the issue >>>>>>>>>>> tracker. In 2007, I became a committer. Most of my initial >>>>>>>>>>> work was >>>>>>>>>>> on the >>>>>>>>>>> UI and some work in the applications (mainly Asset Maintenance >>>>>>>>>>> and >>>>>>>>>>> Work >>>>>>>>>>> Effort). I stayed away from touching the framework code >>>>>>>>>>> because it was >>>>>>>>>>> deep, dark, and scary. >>>>>>>>>>> >>>>>>>>>>> Eventually, I started to understand how the framework code >>>>>>>>>>> works and I >>>>>>>>>>> made some minor modifications. As my understanding grew, I >>>>>>>>>>> progressed >>>>>>>>>>> to >>>>>>>>>>> rewriting large swaths of framework code - making it >>>>>>>>>>> thread-safe, >>>>>>>>>>> fault >>>>>>>>>>> tolerant, efficient, and easier to use. >>>>>>>>>>> >>>>>>>>>>> I will list some of my contributions here, so everyone can >>>>>>>>>>> have a >>>>>>>>>>> clear >>>>>>>>>>> understanding of my experience with the framework code: >>>>>>>>>>> >>>>>>>>>>> New Features >>>>>>>>>>> >>>>>>>>>>> User Preferences >>>>>>>>>>> >>>>>>>>>>> Visual Themes >>>>>>>>>>> >>>>>>>>>>> Custom UI Label XML File Format >>>>>>>>>>> >>>>>>>>>>> Temporal Expressions >>>>>>>>>>> >>>>>>>>>>> Data Type Conversion Framework >>>>>>>>>>> >>>>>>>>>>> Screen Widget Boundary Comments >>>>>>>>>>> >>>>>>>>>>> Metrics >>>>>>>>>>> >>>>>>>>>>> Integrations >>>>>>>>>>> >>>>>>>>>>> UEL >>>>>>>>>>> >>>>>>>>>>> iCalendar >>>>>>>>>>> >>>>>>>>>>> JSR 223 >>>>>>>>>>> >>>>>>>>>>> WebDAV >>>>>>>>>>> >>>>>>>>>>> LDAP >>>>>>>>>>> >>>>>>>>>>> Refactorings/Improvements >>>>>>>>>>> >>>>>>>>>>> FlexibleStringExpander >>>>>>>>>>> >>>>>>>>>>> FlexibleMapExpander >>>>>>>>>>> >>>>>>>>>>> FOP Integration >>>>>>>>>>> >>>>>>>>>>> FreeMarkerWorker >>>>>>>>>>> >>>>>>>>>>> Date-Time Handling >>>>>>>>>>> >>>>>>>>>>> Mini-language >>>>>>>>>>> >>>>>>>>>>> Job Scheduler >>>>>>>>>>> >>>>>>>>>>> In addition, I have performed innumerable framework bug fixes. >>>>>>>>>>> >>>>>>>>>>> So, the contents of this message come from years of experience >>>>>>>>>>> mucking >>>>>>>>>>> about in the framework code. >>>>>>>>>>> >>>>>>>>>>> Okay, let's get started... >>>>>>>>>>> >>>>>>>>>>> Initial Problem Statement >>>>>>>>>>> ------------------------- >>>>>>>>>>> >>>>>>>>>>> In 2009, David Jones started a framework rewrite in a branch: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> https://svn.apache.org/repos/asf/ofbiz/branches/executioncontext20090716 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> At the time, there was some agreement that a rewrite was >>>>>>>>>>> necessary, >>>>>>>>>>> but >>>>>>>>>>> there was disagreement as to how the rewrite should be >>>>>>>>>>> incorporated >>>>>>>>>>> into >>>>>>>>>>> the project: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> https://mail-archives.apache.org/mod_mbox/ofbiz-dev/200908.mbox/%3C455601.62605.qm@...%3E >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> There were concerns that a rewrite would break backward >>>>>>>>>>> compatibility. >>>>>>>>>>> Work on the rewrite branch stopped. Eventually, Jacopo >>>>>>>>>>> suggested the >>>>>>>>>>> community be more accepting of backward-incompatible changes: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> https://mail-archives.apache.org/mod_mbox/ofbiz-dev/201004.mbox/%3cD24F129D-4F9F-444E-84AF-ACA46F49990C@...%3e >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Despite an effort to convince David to proceed with the >>>>>>>>>>> framework >>>>>>>>>>> rewrite, >>>>>>>>>>> he ended up doing it in a separate project: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> http://mail-archives.apache.org/mod_mbox/ofbiz-dev/201104.mbox/%3C07565C88-4023-4D24-93A3-A4906E86F939@...%3E >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> This page describes differences between OFBiz and Moqui, and >>>>>>>>>>> within >>>>>>>>>>> it you >>>>>>>>>>> can extract information on the problems David was trying to >>>>>>>>>>> solve: >>>>>>>>>>> >>>>>>>>>>> http://sourceforge.net/p/moqui/discussion/1086127/thread/4c52f240/ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> There was an email he sent out on the OFBiz dev list where he >>>>>>>>>>> listed >>>>>>>>>>> the >>>>>>>>>>> problems he saw in the framework, but I can't find it. The >>>>>>>>>>> rest of >>>>>>>>>>> this >>>>>>>>>>> message will include the issues he mentioned (the ones I >>>>>>>>>>> remember). I >>>>>>>>>>> was >>>>>>>>>>> in agreement with him at the time, and I still agree that a >>>>>>>>>>> framework >>>>>>>>>>> rewrite is necessary. >>>>>>>>>>> >>>>>>>>>>> The Problems >>>>>>>>>>> ------------ >>>>>>>>>>> >>>>>>>>>>> Code is scattered everywhere - due to an initial effort to >>>>>>>>>>> make the >>>>>>>>>>> framework modular. This causes serious problems. The mere fact >>>>>>>>>>> that >>>>>>>>>>> components like entityext and securityext EXIST makes it clear >>>>>>>>>>> that >>>>>>>>>>> there >>>>>>>>>>> are problems - those components should not be there. Also, >>>>>>>>>>> we run >>>>>>>>>>> into the >>>>>>>>>>> recurring problem of circular dependencies (component A will >>>>>>>>>>> not build >>>>>>>>>>> unless component B is built, and component B will not build >>>>>>>>>>> unless >>>>>>>>>>> component A is built). >>>>>>>>>>> >>>>>>>>>>> Bad separation of concerns. There are far too many examples of >>>>>>>>>>> classes >>>>>>>>>>> that try to be everything to everyone. This makes debugging >>>>>>>>>>> difficult, and >>>>>>>>>>> it makes maintenance/improvements a nightmare. [Using an >>>>>>>>>>> analogy, >>>>>>>>>>> consider >>>>>>>>>>> an automobile design where a spark plug is not separate from a >>>>>>>>>>> transmission. Instead, the automobile uses a >>>>>>>>>>> spark-plug-transmission. >>>>>>>>>>> So >>>>>>>>>>> when the engine is running rough because the spark plug is >>>>>>>>>>> bad, you >>>>>>>>>>> have to >>>>>>>>>>> replace the spark plug AND the transmission.] A good framework >>>>>>>>>>> example can >>>>>>>>>>> be found in my rewrite of the mini-language code. >>>>>>>>>>> Originally, the >>>>>>>>>>> models >>>>>>>>>>> AND the script execution context both contained script >>>>>>>>>>> behaviors - >>>>>>>>>>> making >>>>>>>>>>> debugging/improvements difficult. I changed it so only the >>>>>>>>>>> models >>>>>>>>>>> contain >>>>>>>>>>> script behavior and the script execution context contains only >>>>>>>>>>> the >>>>>>>>>>> script >>>>>>>>>>> execution state. >>>>>>>>>>> >>>>>>>>>>> Lack of good OO design. There are many places where a bit of >>>>>>>>>>> framework >>>>>>>>>>> functionality is contained in a single method that is >>>>>>>>>>> hundreds or >>>>>>>>>>> thousands >>>>>>>>>>> of lines long. There is a term for that: Brittle Code. Code >>>>>>>>>>> isn't >>>>>>>>>>> reused. >>>>>>>>>>> Instead, it is copy-and-pasted all over - so when a problem is >>>>>>>>>>> found >>>>>>>>>>> in the >>>>>>>>>>> C&P code, it has to be fixed in many places instead of one. >>>>>>>>>>> >>>>>>>>>>> Fail-slow design. There are a lot of places in low-level code >>>>>>>>>>> where an >>>>>>>>>>> error condition is encountered, but instead of throwing an >>>>>>>>>>> exception, >>>>>>>>>>> the >>>>>>>>>>> error is ignored and maybe it is logged, or the code tries to >>>>>>>>>>> "guess" >>>>>>>>>>> at a >>>>>>>>>>> solution and then provide an arbitrary default behavior. I've >>>>>>>>>>> seen >>>>>>>>>>> many >>>>>>>>>>> developers struggle with debugging a problem because they >>>>>>>>>>> didn't look >>>>>>>>>>> at >>>>>>>>>>> the logs, or because the error was not logged and there is no >>>>>>>>>>> way of >>>>>>>>>>> knowing what caused it. They end up spending hours >>>>>>>>>>> single-stepping >>>>>>>>>>> through >>>>>>>>>>> code until it reaches the error. >>>>>>>>>>> >>>>>>>>>>> Out-of-date code. A good example is the use of Javolution. That >>>>>>>>>>> library >>>>>>>>>>> was beneficial in the Java 1.4 days, but it is not necessary >>>>>>>>>>> today >>>>>>>>>>> because >>>>>>>>>>> of improved garbage collection. Another good example is DCL >>>>>>>>>>> code. DCL >>>>>>>>>>> was >>>>>>>>>>> used extensively in OFBiz, but it is clearly documented to >>>>>>>>>>> be an >>>>>>>>>>> unreliable >>>>>>>>>>> design (I can get it to fail 90% of the time). Some DCL code >>>>>>>>>>> has been >>>>>>>>>>> replaced, but a lot of it is still there. >>>>>>>>>>> >>>>>>>>>>> Portions of the API are overly complicated. Some methods >>>>>>>>>>> require a >>>>>>>>>>> collection of user-specified artifacts/arguments, which makes >>>>>>>>>>> client >>>>>>>>>>> code >>>>>>>>>>> complicated and verbose. (David solved that problem with his >>>>>>>>>>> Execution >>>>>>>>>>> Context.) Portions of the API are cluttered with unnecessary >>>>>>>>>>> "convenience >>>>>>>>>>> methods" - making the API harder to learn and memorize. In some >>>>>>>>>>> places, a >>>>>>>>>>> domain-specific API is spread across instance methods and >>>>>>>>>>> static >>>>>>>>>>> methods >>>>>>>>>>> and across different classes - making the API hard to >>>>>>>>>>> understand and >>>>>>>>>>> use. >>>>>>>>>>> Yes, there can be good designs that require something like >>>>>>>>>>> that, but >>>>>>>>>>> in the >>>>>>>>>>> OFBiz framework, it exists because of a bad design, not a good >>>>>>>>>>> one. >>>>>>>>>>> >>>>>>>>>>> Use of thread-local variables. This makes multi-threaded design >>>>>>>>>>> impossible. The J2EE specification and the Servlet API require >>>>>>>>>>> one >>>>>>>>>>> thread >>>>>>>>>>> per request (and most J2EE libraries depend on that behavior), >>>>>>>>>>> so the >>>>>>>>>>> current design makes sense from a J2EE perspective, but what >>>>>>>>>>> if I >>>>>>>>>>> don't >>>>>>>>>>> want to run the framework in a J2EE container? Which leads to >>>>>>>>>>> the next >>>>>>>>>>> problem... >>>>>>>>>>> >>>>>>>>>>> Dependence on J2EE designs/APIs/libraries. There are >>>>>>>>>>> developers in the >>>>>>>>>>> Java community (myself included) who are beginning to >>>>>>>>>>> question if >>>>>>>>>>> J2EE is >>>>>>>>>>> really necessary to run web applications. The folks at >>>>>>>>>>> Atomikos are a >>>>>>>>>>> good >>>>>>>>>>> example. OFBiz does not use EJBs, so tying the framework to >>>>>>>>>>> J2EE does >>>>>>>>>>> not >>>>>>>>>>> make sense. It would be better if the framework was designed >>>>>>>>>>> to run >>>>>>>>>>> outside >>>>>>>>>>> a J2EE container, and then have container integration as an >>>>>>>>>>> option. >>>>>>>>>>> >>>>>>>>>>> Configuration files are scattered everywhere. Anyone who has >>>>>>>>>>> deployed >>>>>>>>>>> OFBiz in a production environment will agree this is a >>>>>>>>>>> problem. Try >>>>>>>>>>> changing the HTTP/HTTPS and port settings - it is a nightmare. >>>>>>>>>>> Some >>>>>>>>>>> configuration settings are in nonsensical places. >>>>>>>>>>> >>>>>>>>>>> An abysmal lack of unit testing. I don't have an exact figure >>>>>>>>>>> for code >>>>>>>>>>> coverage, but my gut feeling is coverage is less than 10%. >>>>>>>>>>> Basically, >>>>>>>>>>> we >>>>>>>>>>> all have our fingers crossed - hoping that the framework code >>>>>>>>>>> works as >>>>>>>>>>> expected. This was made painfully obvious a while back when >>>>>>>>>>> I was >>>>>>>>>>> looking >>>>>>>>>>> at some entity caching code and thought to myself "this code >>>>>>>>>>> can't >>>>>>>>>>> work." >>>>>>>>>>> So I wrote some entity cache unit tests and confirmed that the >>>>>>>>>>> entity >>>>>>>>>>> cache >>>>>>>>>>> had serious problems. Think about that - years passed with no >>>>>>>>>>> entity >>>>>>>>>>> cache >>>>>>>>>>> unit tests and consequently we had no idea it wasn't working. >>>>>>>>>>> >>>>>>>>>>> Fix Versus Rewrite >>>>>>>>>>> ------------------ >>>>>>>>>>> >>>>>>>>>>> Jira issues could be created for these problems and teams of >>>>>>>>>>> developers >>>>>>>>>>> could work to fix them. >>>>>>>>>>> >>>>>>>>>>> Or, we could create a branch and start over from scratch. This >>>>>>>>>>> time >>>>>>>>>>> around, there should be less push-back from people concerned >>>>>>>>>>> about >>>>>>>>>>> backwards compatibility. A rewrite offers the advantage of >>>>>>>>>>> reconsidering >>>>>>>>>>> everything - like API design, general problem solving, and new >>>>>>>>>>> features. >>>>>>>>>>> >>>>>>>>>>> I created a Wiki page for a framework design: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> https://cwiki.apache.org/confluence/display/OFBADMIN/Another+Framework+Vision >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> but there hasn't been much interest in it. If the community >>>>>>>>>>> decides >>>>>>>>>>> to go >>>>>>>>>>> ahead with a rewrite, then please feel free to use the Wiki >>>>>>>>>>> pages as a >>>>>>>>>>> guide. >>>>>>>>>>> >>>>>>>>>>> Sandglass Foundation >>>>>>>>>>> -------------------- >>>>>>>>>>> >>>>>>>>>>> Like David, I came to the conclusion that a framework rewrite >>>>>>>>>>> would be >>>>>>>>>>> easier outside the OFBiz community. So, I created my own >>>>>>>>>>> library >>>>>>>>>>> called >>>>>>>>>>> Foundation: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> http://www.sandglass-software.com/products/sandglass/documents/Foundation_Brochure.pdf >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> (PDF) >>>>>>>>>>> >>>>>>>>>>> and I only mention it here to stress how wonderful it can be >>>>>>>>>>> to start >>>>>>>>>>> with >>>>>>>>>>> a clean slate and design an API that is concise yet powerful. >>>>>>>>>>> (Please >>>>>>>>>>> do >>>>>>>>>>> not discuss Foundation here, contact me privately if you want >>>>>>>>>>> more >>>>>>>>>>> information.) >>>>>>>>>>> >>>>>>>>>>> Some examples of what can be done with a rewrite: >>>>>>>>>>> >>>>>>>>>>> A single configuration file >>>>>>>>>>> Use ANSI/ISO SQL SELECT statement strings instead of >>>>>>>>>>> constructing >>>>>>>>>>> complicated Java structures >>>>>>>>>>> Simultaneous asynchronous queries >>>>>>>>>>> Relational integrity across multiple datasources >>>>>>>>>>> Multi-table SELECT across multiple datasources >>>>>>>>>>> Automatic and transparent row version control >>>>>>>>>>> Automatic and transparent multi-language datasource >>>>>>>>>>> support >>>>>>>>>>> Abstract entities (similar to SQL user types) >>>>>>>>>>> Service engine throttling (protects against server >>>>>>>>>>> over-utilization) >>>>>>>>>>> Simplified security (authorization) ( >>>>>>>>>>> >>>>>>>>>>> https://cwiki.apache.org/confluence/display/OFBIZ/OFBiz+Security+Redesign) >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Pure interface-based API - so developers are free to >>>>>>>>>>> modify >>>>>>>>>>> framework >>>>>>>>>>> behavior by using decorators >>>>>>>>>>> Thorough unit tests >>>>>>>>>>> >>>>>>>>>>> Benefits of a rewrite: >>>>>>>>>>> >>>>>>>>>>> Reduced resource requirements (lower hosting fees) >>>>>>>>>>> Reduce application development time - due to a >>>>>>>>>>> simplified API >>>>>>>>>>> Easier framework code maintenance >>>>>>>>>>> Better reliability >>>>>>>>>>> >>>>>>>>>>> Conclusion >>>>>>>>>>> ---------- >>>>>>>>>>> >>>>>>>>>>> Like I said at the start, this is all I will say about the >>>>>>>>>>> subject. >>>>>>>>>>> I'm >>>>>>>>>>> done trying to convince everyone. I hope someone agrees with >>>>>>>>>>> me and >>>>>>>>>>> they >>>>>>>>>>> are able to build support for the idea. >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Adrian Crum >>>>>>>>>>> Sandglass Software >>>>>>>>>>> www.sandglass-software.com >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>> >> >> |
Re: Why A Framework Rewrite Is Necessary
|
|
In reply to this post by Pierre Smits
> On 16 Oct 2015, at 01:41, Pierre Smits <[hidden email]> wrote: > > I understand the reluctance of the community, because the impact will be > huge. When looking at the data in OpenHub I see OFBiz having an estimate > effort spend of 519 person years vs 6 for the combined > Moqui-Mantle-HiveMindPM-PopCommerce suite. And one of the reasons behind it > is simple: Many more have worked on OFBiz (from day 1) than on the Moqui > suite. One could even argue that both directions took the same number of > years in duration to get where they are now. Without all the experiences > regarding the OFBiz product there couldn't have been an evolution called > the Moqui suite. Quick note on this: the OpenHub stats for Moqui are messed up somehow, ie the code analysis is messed up showing around negative 21k lines of code (the real number is around 60k lines of code, 75k lines of text), so the COCOMO estimate is way off. That said, for the same functionality fewer lines of code is easier to maintain and IMO OFBiz could do a lot better in this respect, ie the same functionality in probably 20-25% of the lines of code. There is a lot of unused an inefficient code in OFBiz, and overall it would be more valuable if the code size were smaller (not so much to end users, but to contributors, committers, customizers, etc). Your point about Moqui not being possible without OFBiz is absolutely true. Most of the ideas for early versions of Moqui/Mantle came from OFBiz and in fact it was the ever growing list of things that would be really nice in OFBiz that were the impetus for even starting Moqui/Mantle/etc. OFBiz itself has huge value as it is, even if there is big room for improvement and innovation. Unfortunately there are barriers to that as well, including backward compatibility which for many cleanups and innovations just has to go. That is why this discussion of a fresh implementation of the framework in OFBiz is a big step. More comments on that in another reply… the idea of defining an API and XSDs for the various XML files as a first step to a fresh framework implementation is a very good one. -David |
Re: Why A Framework Rewrite Is Necessary
|
|
In reply to this post by Ron Wheeler
This message has a lot of the right questions to ask about a new framework. A big part of why the OFBiz framework suffers is because it was never planned in its entirety from the beginning, it is VERY much an emergent design as opposed to an intentional one. The interest bits start appearing with intentional design, and for anyone serious about getting into a fresh framework rewrite I’d highly recommend doing the same thing I did with Moqui: define Java interfaces for the API and an XML schema (XSD files) for the intended XML files. The XML schema might be the same as the current one, though there is room for improvement in the current OFBiz XML files and the eventual framework would be much better if these are reconsidered as well. You can see the Moqui interfaces here: https://github.com/moqui/moqui/tree/master/framework/src/api/java/org/moqui The central object at runtime, ie for most application code, is the implementation of the ExecutionContext interface. The whole API is structured around this: https://github.com/moqui/moqui/blob/master/framework/src/api/java/org/moqui/context/ExecutionContext.java There is also a JavaDoc generated and available on moqui.org, might be easier to read for some: http://www.moqui.org/javadoc/index.html http://www.moqui.org/javadoc/org/moqui/context/ExecutionContext.html I include these links because they might be useful for ideas or even as a starting point for a next generation OFBiz framework API (take or leave each method/etc as you wish). The nice thing about creating Java interfaces as opposed to a document is they are more clear and concise, and can be actually used in the framework implementation once interested community participants have reviewed it. This doesn’t require nearly as much work as implementing the beast and is a great way to communicate concepts, so for anyone really serious about a framework rewrite in OFBiz I’d highly recommend this specific effort. The other details like which tools (other open source projects and such) to use is less important. On the other hand if using something like Spring is a serious consideration it would change the API dramatically, and a lot of the design ideas as well (I’d recommend against this BTW, the Spring ecosystem is its own thing and VERY different conceptually from OFBiz). -David > On 16 Oct 2015, at 08:47, Ron Wheeler <[hidden email]> wrote: > > Before we start to decide who can access code, we need to decide on a roadmap for the new framework. > How different will the API be from the current framework in each of the areas that the framework will offer services? > > How modular will it be? > Foundation identifies > - Core with 3 main areas of functionality. Can they be separated into separate projects with clean interfaces? are there more projects such as authentication, persistence, logging and audit (see below) that are shared across the Foundation Core high level features. > - Script > - Imex > - Connect - seems to a number of projects here that could be tackled separately. > Is this it? > > Will there be an application isolation layer that will support OFBiz's current interaction with the Framework. This should also be a separate project where OFBiz knowledge is really valuable. > > What will go underneath the covers? Spring-Boot , Spring JPA, Hibernate, etc. > > How many containers will be supported. Tomcat, Jetty, Glassfish, Spring-Boot. > > How many persistence options will be supported? SQL, No-SQL > > How many authentication services will be supported - internal, LDAP, Oauth, Google, LinkedIn, Facebook. > > What administration functions will be offered? How? JConsole, REST, browser/mobile apps. > > How delivered? Installer, Docker image, VM image, > > What demo apps? > > What test framework(s)? Test Applications. > > What would be a reasonable set of functionality to be released in version 1.0? Minimum useful framework. > > How many people would it take to do this in a reasonable timeframe? > > Ron > > > On 16/10/2015 3:41 AM, Pierre Smits wrote: >> On Fri, Oct 16, 2015 at 5:31 AM, David E. Jones <[hidden email]> wrote: >> >>> >>>> On 15 Oct 2015, at 07:58, Adrian Crum < >>> [hidden email]> wrote: >>>> Keep in mind that much of David's code in OFBiz has been rewritten. So >>> yes, we CAN do a better job than him. >>> >>> I think there’s a name for this logical fallacy… >>> >>> And this could also be called a logical fallacy. But let us not make it >> into a pissing contest again, like we had in the past regarding the >> opposing viewpoints on this. >> >> >>>> Also keep in mind that Moqui duplicates some of the problems I listed - >>> so by using Moqui, we keep the same problems instead of fixing them. >>> >>> Could you be more specific, other than the type conversion stuff you >>> mentioned many years ago (which I fully disagree with)? >>> >>> This is not about who is right or wrong, but where the community wants to >> go. >> >> I understand the reluctance of the community, because the impact will be >> huge. When looking at the data in OpenHub I see OFBiz having an estimate >> effort spend of 519 person years vs 6 for the combined >> Moqui-Mantle-HiveMindPM-PopCommerce suite. And one of the reasons behind it >> is simple: Many more have worked on OFBiz (from day 1) than on the Moqui >> suite. One could even argue that both directions took the same number of >> years in duration to get where they are now. Without all the experiences >> regarding the OFBiz product there couldn't have been an evolution called >> the Moqui suite. >> >> Coming back to opting for a new direction, as Scott has stated we can have >> this in a separate code repository (subproject, like many other Apache >> projects do their work) even combined with a new JIRA an Wiki under the >> umbrella of the OFBiz project. Based on the comments provided, this seems >> like a logical choice to ensure that adopters of the current solution will >> keep the support of the community while at the same time ensuring >> containment of the new approach. >> >> But these are mere technical, supportive aspects. The more important issue >> is what this new solution will encompass. There are talks of a rewrite, >> which sounds like reinvention of the wheel. But I guess it is not like >> that. Yet, taking decisions based on a few one-pagers (e.g. >> http://www.sandglass-software.com/products/sandglass/documents/Foundation_Brochure.pdf) >> are seldom done. Maybe it works for a single person, but I doubt it will >> make a community fly. >> >> Whether fix or rewrite, choices will be made regarding the supporting 3rd >> party libraries/solutions and the community needs to understand the impact >> to get behind it. So before we embark on the coding trip, let us get into >> trying to define a bit more what the new solution will encompass and get >> consensus on that. >> >> Another issue regarding getting the community behind behind this new effort >> is this: 'restrict access to the new code'. I guess this meant restrict >> write access. Though understandable from a avoidance of dilution/scope >> creep point of view, I see this as a wrong direction. This 'proposed' >> exclusion of contributors of the kind will bring us back to where this >> project came from: discrimination and favouritism. I even doubt that this >> is possible under the current principles of the ASF. >> Given that this is an enormous undertaking, we need to get as many on board >> as possible. Not only to ensure that the decisions (at each level) will get >> consensus, but also the ensure that every aspect down the line will get >> addressed (e.g. documentation, test definitions, marketing/promotions, etc) >> in order to get this kite flying. >> >> Best regards, >> >> Pierre Smits >> >> *OFBiz Extensions Marketplace* >> http://oem.ofbizci.net/oci-2/ >> > > > -- > Ron Wheeler > President > Artifact Software Inc > email: [hidden email] > skype: ronaldmwheeler > phone: 866-970-2435, ext 102 > |
Re: Why A Framework Rewrite Is Necessary
|
|
Thanks David! I agree with most of it, but... (inline)
Adrian Crum Sandglass Software www.sandglass-software.com On 10/17/2015 11:07 AM, David E. Jones wrote: > > This message has a lot of the right questions to ask about a new framework. > > A big part of why the OFBiz framework suffers is because it was never planned in its entirety from the beginning, it is VERY much an emergent design as opposed to an intentional one. > > The interest bits start appearing with intentional design, and for anyone serious about getting into a fresh framework rewrite I’d highly recommend doing the same thing I did with Moqui: define Java interfaces for the API and an XML schema (XSD files) for the intended XML files. The XML schema might be the same as the current one, though there is room for improvement in the current OFBiz XML files and the eventual framework would be much better if these are reconsidered as well. > > You can see the Moqui interfaces here: > > https://github.com/moqui/moqui/tree/master/framework/src/api/java/org/moqui > > The central object at runtime, ie for most application code, is the implementation of the ExecutionContext interface. The whole API is structured around this: > > https://github.com/moqui/moqui/blob/master/framework/src/api/java/org/moqui/context/ExecutionContext.java > > There is also a JavaDoc generated and available on moqui.org, might be easier to read for some: > > http://www.moqui.org/javadoc/index.html > http://www.moqui.org/javadoc/org/moqui/context/ExecutionContext.html > > I include these links because they might be useful for ideas or even as a starting point for a next generation OFBiz framework API (take or leave each method/etc as you wish). > > The nice thing about creating Java interfaces as opposed to a document is they are more clear and concise, and can be actually used in the framework implementation once interested community participants have reviewed it. I believe both are necessary. The design document Wiki page describes the thought process/design goals/use cases for the interface design, and the actual Java interface can be included for review. Not only is this good for the design process, it can also serve as a reference for later ("Why was the interface designed that way?"). Here is an example: https://cwiki.apache.org/confluence/display/OFBIZ/Authorization+Manager+Implementation+Details > > This doesn’t require nearly as much work as implementing the beast and is a great way to communicate concepts, so for anyone really serious about a framework rewrite in OFBiz I’d highly recommend this specific effort. > > The other details like which tools (other open source projects and such) to use is less important. On the other hand if using something like Spring is a serious consideration it would change the API dramatically, and a lot of the design ideas as well (I’d recommend against this BTW, the Spring ecosystem is its own thing and VERY different conceptually from OFBiz). > > -David |
Re: Why A Framework Rewrite Is Necessary
|
|
In reply to this post by David E. Jones-2
This seems like a very sound approach and something that can be started
in the wiki. That way no one will be tempted to start coding before we know what to code. If the first things to be coded (even if the wiki is the repo) are interfaces and XML schemas, we should be able to divide up the implementation classes among a wide number of committers while still maintaining a sense of confidence that the result will be what was agreed upon. It will also allow us to validate: 1) Is there a consensus about the design and function of a new framework 2) Is there enough of a community to actually build one. Can we start with the pages that Adrian wrote way back then, as a basic outline of the components of a framework? https://cwiki.apache.org/confluence/display/OFBADMIN/Another+Framework+Vision Ron On 17/10/2015 2:07 PM, David E. Jones wrote: > This message has a lot of the right questions to ask about a new framework. > > A big part of why the OFBiz framework suffers is because it was never planned in its entirety from the beginning, it is VERY much an emergent design as opposed to an intentional one. > > The interest bits start appearing with intentional design, and for anyone serious about getting into a fresh framework rewrite I’d highly recommend doing the same thing I did with Moqui: define Java interfaces for the API and an XML schema (XSD files) for the intended XML files. The XML schema might be the same as the current one, though there is room for improvement in the current OFBiz XML files and the eventual framework would be much better if these are reconsidered as well. > > You can see the Moqui interfaces here: > > https://github.com/moqui/moqui/tree/master/framework/src/api/java/org/moqui > > The central object at runtime, ie for most application code, is the implementation of the ExecutionContext interface. The whole API is structured around this: > > https://github.com/moqui/moqui/blob/master/framework/src/api/java/org/moqui/context/ExecutionContext.java > > There is also a JavaDoc generated and available on moqui.org, might be easier to read for some: > > http://www.moqui.org/javadoc/index.html > http://www.moqui.org/javadoc/org/moqui/context/ExecutionContext.html > > I include these links because they might be useful for ideas or even as a starting point for a next generation OFBiz framework API (take or leave each method/etc as you wish). > > The nice thing about creating Java interfaces as opposed to a document is they are more clear and concise, and can be actually used in the framework implementation once interested community participants have reviewed it. > > This doesn’t require nearly as much work as implementing the beast and is a great way to communicate concepts, so for anyone really serious about a framework rewrite in OFBiz I’d highly recommend this specific effort. > > The other details like which tools (other open source projects and such) to use is less important. On the other hand if using something like Spring is a serious consideration it would change the API dramatically, and a lot of the design ideas as well (I’d recommend against this BTW, the Spring ecosystem is its own thing and VERY different conceptually from OFBiz). > > -David > > >> On 16 Oct 2015, at 08:47, Ron Wheeler <[hidden email]> wrote: >> >> Before we start to decide who can access code, we need to decide on a roadmap for the new framework. >> How different will the API be from the current framework in each of the areas that the framework will offer services? >> >> How modular will it be? >> Foundation identifies >> - Core with 3 main areas of functionality. Can they be separated into separate projects with clean interfaces? are there more projects such as authentication, persistence, logging and audit (see below) that are shared across the Foundation Core high level features. >> - Script >> - Imex >> - Connect - seems to a number of projects here that could be tackled separately. >> Is this it? >> >> Will there be an application isolation layer that will support OFBiz's current interaction with the Framework. This should also be a separate project where OFBiz knowledge is really valuable. >> >> What will go underneath the covers? Spring-Boot , Spring JPA, Hibernate, etc. >> >> How many containers will be supported. Tomcat, Jetty, Glassfish, Spring-Boot. >> >> How many persistence options will be supported? SQL, No-SQL >> >> How many authentication services will be supported - internal, LDAP, Oauth, Google, LinkedIn, Facebook. >> >> What administration functions will be offered? How? JConsole, REST, browser/mobile apps. >> >> How delivered? Installer, Docker image, VM image, >> >> What demo apps? >> >> What test framework(s)? Test Applications. >> >> What would be a reasonable set of functionality to be released in version 1.0? Minimum useful framework. >> >> How many people would it take to do this in a reasonable timeframe? >> >> Ron >> >> >> On 16/10/2015 3:41 AM, Pierre Smits wrote: >>> On Fri, Oct 16, 2015 at 5:31 AM, David E. Jones <[hidden email]> wrote: >>> >>>>> On 15 Oct 2015, at 07:58, Adrian Crum < >>>> [hidden email]> wrote: >>>>> Keep in mind that much of David's code in OFBiz has been rewritten. So >>>> yes, we CAN do a better job than him. >>>> >>>> I think there’s a name for this logical fallacy… >>>> >>>> And this could also be called a logical fallacy. But let us not make it >>> into a pissing contest again, like we had in the past regarding the >>> opposing viewpoints on this. >>> >>> >>>>> Also keep in mind that Moqui duplicates some of the problems I listed - >>>> so by using Moqui, we keep the same problems instead of fixing them. >>>> >>>> Could you be more specific, other than the type conversion stuff you >>>> mentioned many years ago (which I fully disagree with)? >>>> >>>> This is not about who is right or wrong, but where the community wants to >>> go. >>> >>> I understand the reluctance of the community, because the impact will be >>> huge. When looking at the data in OpenHub I see OFBiz having an estimate >>> effort spend of 519 person years vs 6 for the combined >>> Moqui-Mantle-HiveMindPM-PopCommerce suite. And one of the reasons behind it >>> is simple: Many more have worked on OFBiz (from day 1) than on the Moqui >>> suite. One could even argue that both directions took the same number of >>> years in duration to get where they are now. Without all the experiences >>> regarding the OFBiz product there couldn't have been an evolution called >>> the Moqui suite. >>> >>> Coming back to opting for a new direction, as Scott has stated we can have >>> this in a separate code repository (subproject, like many other Apache >>> projects do their work) even combined with a new JIRA an Wiki under the >>> umbrella of the OFBiz project. Based on the comments provided, this seems >>> like a logical choice to ensure that adopters of the current solution will >>> keep the support of the community while at the same time ensuring >>> containment of the new approach. >>> >>> But these are mere technical, supportive aspects. The more important issue >>> is what this new solution will encompass. There are talks of a rewrite, >>> which sounds like reinvention of the wheel. But I guess it is not like >>> that. Yet, taking decisions based on a few one-pagers (e.g. >>> http://www.sandglass-software.com/products/sandglass/documents/Foundation_Brochure.pdf) >>> are seldom done. Maybe it works for a single person, but I doubt it will >>> make a community fly. >>> >>> Whether fix or rewrite, choices will be made regarding the supporting 3rd >>> party libraries/solutions and the community needs to understand the impact >>> to get behind it. So before we embark on the coding trip, let us get into >>> trying to define a bit more what the new solution will encompass and get >>> consensus on that. >>> >>> Another issue regarding getting the community behind behind this new effort >>> is this: 'restrict access to the new code'. I guess this meant restrict >>> write access. Though understandable from a avoidance of dilution/scope >>> creep point of view, I see this as a wrong direction. This 'proposed' >>> exclusion of contributors of the kind will bring us back to where this >>> project came from: discrimination and favouritism. I even doubt that this >>> is possible under the current principles of the ASF. >>> Given that this is an enormous undertaking, we need to get as many on board >>> as possible. Not only to ensure that the decisions (at each level) will get >>> consensus, but also the ensure that every aspect down the line will get >>> addressed (e.g. documentation, test definitions, marketing/promotions, etc) >>> in order to get this kite flying. >>> >>> Best regards, >>> >>> Pierre Smits >>> >>> *OFBiz Extensions Marketplace* >>> http://oem.ofbizci.net/oci-2/ >>> >> >> -- >> Ron Wheeler >> President >> Artifact Software Inc >> email: [hidden email] >> skype: ronaldmwheeler >> phone: 866-970-2435, ext 102 >> > -- Ron Wheeler President Artifact Software Inc email: [hidden email] skype: ronaldmwheeler phone: 866-970-2435, ext 102 |
|
|
Hello Everyone,
Like Ron I also find the suggestion from David logical and sensible. We can perhaps start with a design for interfaces and XSDs as a first step. Would anyone like to join forces to brainstorm on this issue? Adrian, is that a route that you also prefer? Taher Alkhateeb ----- Original Message ----- From: "Ron Wheeler" <[hidden email]> To: [hidden email] Sent: Saturday, 17 October, 2015 10:39:35 PM Subject: Re: Why A Framework Rewrite Is Necessary This seems like a very sound approach and something that can be started in the wiki. That way no one will be tempted to start coding before we know what to code. If the first things to be coded (even if the wiki is the repo) are interfaces and XML schemas, we should be able to divide up the implementation classes among a wide number of committers while still maintaining a sense of confidence that the result will be what was agreed upon. It will also allow us to validate: 1) Is there a consensus about the design and function of a new framework 2) Is there enough of a community to actually build one. Can we start with the pages that Adrian wrote way back then, as a basic outline of the components of a framework? https://cwiki.apache.org/confluence/display/OFBADMIN/Another+Framework+Vision Ron On 17/10/2015 2:07 PM, David E. Jones wrote: > This message has a lot of the right questions to ask about a new framework. > > A big part of why the OFBiz framework suffers is because it was never planned in its entirety from the beginning, it is VERY much an emergent design as opposed to an intentional one. > > The interest bits start appearing with intentional design, and for anyone serious about getting into a fresh framework rewrite I’d highly recommend doing the same thing I did with Moqui: define Java interfaces for the API and an XML schema (XSD files) for the intended XML files. The XML schema might be the same as the current one, though there is room for improvement in the current OFBiz XML files and the eventual framework would be much better if these are reconsidered as well. > > You can see the Moqui interfaces here: > > https://github.com/moqui/moqui/tree/master/framework/src/api/java/org/moqui > > The central object at runtime, ie for most application code, is the implementation of the ExecutionContext interface. The whole API is structured around this: > > https://github.com/moqui/moqui/blob/master/framework/src/api/java/org/moqui/context/ExecutionContext.java > > There is also a JavaDoc generated and available on moqui.org, might be easier to read for some: > > http://www.moqui.org/javadoc/index.html > http://www.moqui.org/javadoc/org/moqui/context/ExecutionContext.html > > I include these links because they might be useful for ideas or even as a starting point for a next generation OFBiz framework API (take or leave each method/etc as you wish). > > The nice thing about creating Java interfaces as opposed to a document is they are more clear and concise, and can be actually used in the framework implementation once interested community participants have reviewed it. > > This doesn’t require nearly as much work as implementing the beast and is a great way to communicate concepts, so for anyone really serious about a framework rewrite in OFBiz I’d highly recommend this specific effort. > > The other details like which tools (other open source projects and such) to use is less important. On the other hand if using something like Spring is a serious consideration it would change the API dramatically, and a lot of the design ideas as well (I’d recommend against this BTW, the Spring ecosystem is its own thing and VERY different conceptually from OFBiz). > > -David > > >> On 16 Oct 2015, at 08:47, Ron Wheeler <[hidden email]> wrote: >> >> Before we start to decide who can access code, we need to decide on a roadmap for the new framework. >> How different will the API be from the current framework in each of the areas that the framework will offer services? >> >> How modular will it be? >> Foundation identifies >> - Core with 3 main areas of functionality. Can they be separated into separate projects with clean interfaces? are there more projects such as authentication, persistence, logging and audit (see below) that are shared across the Foundation Core high level features. >> - Script >> - Imex >> - Connect - seems to a number of projects here that could be tackled separately. >> Is this it? >> >> Will there be an application isolation layer that will support OFBiz's current interaction with the Framework. This should also be a separate project where OFBiz knowledge is really valuable. >> >> What will go underneath the covers? Spring-Boot , Spring JPA, Hibernate, etc. >> >> How many containers will be supported. Tomcat, Jetty, Glassfish, Spring-Boot. >> >> How many persistence options will be supported? SQL, No-SQL >> >> How many authentication services will be supported - internal, LDAP, Oauth, Google, LinkedIn, Facebook. >> >> What administration functions will be offered? How? JConsole, REST, browser/mobile apps. >> >> How delivered? Installer, Docker image, VM image, >> >> What demo apps? >> >> What test framework(s)? Test Applications. >> >> What would be a reasonable set of functionality to be released in version 1.0? Minimum useful framework. >> >> How many people would it take to do this in a reasonable timeframe? >> >> Ron >> >> >> On 16/10/2015 3:41 AM, Pierre Smits wrote: >>> On Fri, Oct 16, 2015 at 5:31 AM, David E. Jones <[hidden email]> wrote: >>> >>>>> On 15 Oct 2015, at 07:58, Adrian Crum < >>>> [hidden email]> wrote: >>>>> Keep in mind that much of David's code in OFBiz has been rewritten. So >>>> yes, we CAN do a better job than him. >>>> >>>> I think there’s a name for this logical fallacy… >>>> >>>> And this could also be called a logical fallacy. But let us not make it >>> into a pissing contest again, like we had in the past regarding the >>> opposing viewpoints on this. >>> >>> >>>>> Also keep in mind that Moqui duplicates some of the problems I listed - >>>> so by using Moqui, we keep the same problems instead of fixing them. >>>> >>>> Could you be more specific, other than the type conversion stuff you >>>> mentioned many years ago (which I fully disagree with)? >>>> >>>> This is not about who is right or wrong, but where the community wants to >>> go. >>> >>> I understand the reluctance of the community, because the impact will be >>> huge. When looking at the data in OpenHub I see OFBiz having an estimate >>> effort spend of 519 person years vs 6 for the combined >>> Moqui-Mantle-HiveMindPM-PopCommerce suite. And one of the reasons behind it >>> is simple: Many more have worked on OFBiz (from day 1) than on the Moqui >>> suite. One could even argue that both directions took the same number of >>> years in duration to get where they are now. Without all the experiences >>> regarding the OFBiz product there couldn't have been an evolution called >>> the Moqui suite. >>> >>> Coming back to opting for a new direction, as Scott has stated we can have >>> this in a separate code repository (subproject, like many other Apache >>> projects do their work) even combined with a new JIRA an Wiki under the >>> umbrella of the OFBiz project. Based on the comments provided, this seems >>> like a logical choice to ensure that adopters of the current solution will >>> keep the support of the community while at the same time ensuring >>> containment of the new approach. >>> >>> But these are mere technical, supportive aspects. The more important issue >>> is what this new solution will encompass. There are talks of a rewrite, >>> which sounds like reinvention of the wheel. But I guess it is not like >>> that. Yet, taking decisions based on a few one-pagers (e.g. >>> http://www.sandglass-software.com/products/sandglass/documents/Foundation_Brochure.pdf) >>> are seldom done. Maybe it works for a single person, but I doubt it will >>> make a community fly. >>> >>> Whether fix or rewrite, choices will be made regarding the supporting 3rd >>> party libraries/solutions and the community needs to understand the impact >>> to get behind it. So before we embark on the coding trip, let us get into >>> trying to define a bit more what the new solution will encompass and get >>> consensus on that. >>> >>> Another issue regarding getting the community behind behind this new effort >>> is this: 'restrict access to the new code'. I guess this meant restrict >>> write access. Though understandable from a avoidance of dilution/scope >>> creep point of view, I see this as a wrong direction. This 'proposed' >>> exclusion of contributors of the kind will bring us back to where this >>> project came from: discrimination and favouritism. I even doubt that this >>> is possible under the current principles of the ASF. >>> Given that this is an enormous undertaking, we need to get as many on board >>> as possible. Not only to ensure that the decisions (at each level) will get >>> consensus, but also the ensure that every aspect down the line will get >>> addressed (e.g. documentation, test definitions, marketing/promotions, etc) >>> in order to get this kite flying. >>> >>> Best regards, >>> >>> Pierre Smits >>> >>> *OFBiz Extensions Marketplace* >>> http://oem.ofbizci.net/oci-2/ >>> >> >> -- >> Ron Wheeler >> President >> Artifact Software Inc >> email: [hidden email] >> skype: ronaldmwheeler >> phone: 866-970-2435, ext 102 >> > -- Ron Wheeler President Artifact Software Inc email: [hidden email] skype: ronaldmwheeler phone: 866-970-2435, ext 102 |
Re: Why A Framework Rewrite Is Necessary
|
|
That sounds fine to me.
Keep in mind that Sharan is spearheading this thing, and I will be effectively uninvolved. But I will offer advice/guidance where I feel it is necessary. Based on Ron's comments, I think it would be best if the Wiki page was moved so others can modify it, or C&P it to a less restrictive area. Adrian Crum Sandglass Software www.sandglass-software.com On 10/18/2015 2:21 PM, Taher Alkhateeb wrote: > Hello Everyone, > > Like Ron I also find the suggestion from David logical and sensible. We can perhaps start with a design for interfaces and XSDs as a first step. Would anyone like to join forces to brainstorm on this issue? Adrian, is that a route that you also prefer? > > Taher Alkhateeb > > ----- Original Message ----- > > From: "Ron Wheeler" <[hidden email]> > To: [hidden email] > Sent: Saturday, 17 October, 2015 10:39:35 PM > Subject: Re: Why A Framework Rewrite Is Necessary > > This seems like a very sound approach and something that can be started > in the wiki. > That way no one will be tempted to start coding before we know what to code. > > > If the first things to be coded (even if the wiki is the repo) are > interfaces and XML schemas, we should be able to divide up the > implementation classes among a wide number of committers while still > maintaining a sense of confidence that the result will be what was > agreed upon. > > It will also allow us to validate: > 1) Is there a consensus about the design and function of a new framework > 2) Is there enough of a community to actually build one. > > Can we start with the pages that Adrian wrote way back then, as a basic > outline of the components of a framework? > https://cwiki.apache.org/confluence/display/OFBADMIN/Another+Framework+Vision > > > Ron > > > On 17/10/2015 2:07 PM, David E. Jones wrote: >> This message has a lot of the right questions to ask about a new framework. >> >> A big part of why the OFBiz framework suffers is because it was never planned in its entirety from the beginning, it is VERY much an emergent design as opposed to an intentional one. >> >> The interest bits start appearing with intentional design, and for anyone serious about getting into a fresh framework rewrite I’d highly recommend doing the same thing I did with Moqui: define Java interfaces for the API and an XML schema (XSD files) for the intended XML files. The XML schema might be the same as the current one, though there is room for improvement in the current OFBiz XML files and the eventual framework would be much better if these are reconsidered as well. >> >> You can see the Moqui interfaces here: >> >> https://github.com/moqui/moqui/tree/master/framework/src/api/java/org/moqui >> >> The central object at runtime, ie for most application code, is the implementation of the ExecutionContext interface. The whole API is structured around this: >> >> https://github.com/moqui/moqui/blob/master/framework/src/api/java/org/moqui/context/ExecutionContext.java >> >> There is also a JavaDoc generated and available on moqui.org, might be easier to read for some: >> >> http://www.moqui.org/javadoc/index.html >> http://www.moqui.org/javadoc/org/moqui/context/ExecutionContext.html >> >> I include these links because they might be useful for ideas or even as a starting point for a next generation OFBiz framework API (take or leave each method/etc as you wish). >> >> The nice thing about creating Java interfaces as opposed to a document is they are more clear and concise, and can be actually used in the framework implementation once interested community participants have reviewed it. >> >> This doesn’t require nearly as much work as implementing the beast and is a great way to communicate concepts, so for anyone really serious about a framework rewrite in OFBiz I’d highly recommend this specific effort. >> >> The other details like which tools (other open source projects and such) to use is less important. On the other hand if using something like Spring is a serious consideration it would change the API dramatically, and a lot of the design ideas as well (I’d recommend against this BTW, the Spring ecosystem is its own thing and VERY different conceptually from OFBiz). >> >> -David >> >> >>> On 16 Oct 2015, at 08:47, Ron Wheeler <[hidden email]> wrote: >>> >>> Before we start to decide who can access code, we need to decide on a roadmap for the new framework. >>> How different will the API be from the current framework in each of the areas that the framework will offer services? >>> >>> How modular will it be? >>> Foundation identifies >>> - Core with 3 main areas of functionality. Can they be separated into separate projects with clean interfaces? are there more projects such as authentication, persistence, logging and audit (see below) that are shared across the Foundation Core high level features. >>> - Script >>> - Imex >>> - Connect - seems to a number of projects here that could be tackled separately. >>> Is this it? >>> >>> Will there be an application isolation layer that will support OFBiz's current interaction with the Framework. This should also be a separate project where OFBiz knowledge is really valuable. >>> >>> What will go underneath the covers? Spring-Boot , Spring JPA, Hibernate, etc. >>> >>> How many containers will be supported. Tomcat, Jetty, Glassfish, Spring-Boot. >>> >>> How many persistence options will be supported? SQL, No-SQL >>> >>> How many authentication services will be supported - internal, LDAP, Oauth, Google, LinkedIn, Facebook. >>> >>> What administration functions will be offered? How? JConsole, REST, browser/mobile apps. >>> >>> How delivered? Installer, Docker image, VM image, >>> >>> What demo apps? >>> >>> What test framework(s)? Test Applications. >>> >>> What would be a reasonable set of functionality to be released in version 1.0? Minimum useful framework. >>> >>> How many people would it take to do this in a reasonable timeframe? >>> >>> Ron >>> >>> >>> On 16/10/2015 3:41 AM, Pierre Smits wrote: >>>> On Fri, Oct 16, 2015 at 5:31 AM, David E. Jones <[hidden email]> wrote: >>>> >>>>>> On 15 Oct 2015, at 07:58, Adrian Crum < >>>>> [hidden email]> wrote: >>>>>> Keep in mind that much of David's code in OFBiz has been rewritten. So >>>>> yes, we CAN do a better job than him. >>>>> >>>>> I think there’s a name for this logical fallacy… >>>>> >>>>> And this could also be called a logical fallacy. But let us not make it >>>> into a pissing contest again, like we had in the past regarding the >>>> opposing viewpoints on this. >>>> >>>> >>>>>> Also keep in mind that Moqui duplicates some of the problems I listed - >>>>> so by using Moqui, we keep the same problems instead of fixing them. >>>>> >>>>> Could you be more specific, other than the type conversion stuff you >>>>> mentioned many years ago (which I fully disagree with)? >>>>> >>>>> This is not about who is right or wrong, but where the community wants to >>>> go. >>>> >>>> I understand the reluctance of the community, because the impact will be >>>> huge. When looking at the data in OpenHub I see OFBiz having an estimate >>>> effort spend of 519 person years vs 6 for the combined >>>> Moqui-Mantle-HiveMindPM-PopCommerce suite. And one of the reasons behind it >>>> is simple: Many more have worked on OFBiz (from day 1) than on the Moqui >>>> suite. One could even argue that both directions took the same number of >>>> years in duration to get where they are now. Without all the experiences >>>> regarding the OFBiz product there couldn't have been an evolution called >>>> the Moqui suite. >>>> >>>> Coming back to opting for a new direction, as Scott has stated we can have >>>> this in a separate code repository (subproject, like many other Apache >>>> projects do their work) even combined with a new JIRA an Wiki under the >>>> umbrella of the OFBiz project. Based on the comments provided, this seems >>>> like a logical choice to ensure that adopters of the current solution will >>>> keep the support of the community while at the same time ensuring >>>> containment of the new approach. >>>> >>>> But these are mere technical, supportive aspects. The more important issue >>>> is what this new solution will encompass. There are talks of a rewrite, >>>> which sounds like reinvention of the wheel. But I guess it is not like >>>> that. Yet, taking decisions based on a few one-pagers (e.g. >>>> http://www.sandglass-software.com/products/sandglass/documents/Foundation_Brochure.pdf) >>>> are seldom done. Maybe it works for a single person, but I doubt it will >>>> make a community fly. >>>> >>>> Whether fix or rewrite, choices will be made regarding the supporting 3rd >>>> party libraries/solutions and the community needs to understand the impact >>>> to get behind it. So before we embark on the coding trip, let us get into >>>> trying to define a bit more what the new solution will encompass and get >>>> consensus on that. >>>> >>>> Another issue regarding getting the community behind behind this new effort >>>> is this: 'restrict access to the new code'. I guess this meant restrict >>>> write access. Though understandable from a avoidance of dilution/scope >>>> creep point of view, I see this as a wrong direction. This 'proposed' >>>> exclusion of contributors of the kind will bring us back to where this >>>> project came from: discrimination and favouritism. I even doubt that this >>>> is possible under the current principles of the ASF. >>>> Given that this is an enormous undertaking, we need to get as many on board >>>> as possible. Not only to ensure that the decisions (at each level) will get >>>> consensus, but also the ensure that every aspect down the line will get >>>> addressed (e.g. documentation, test definitions, marketing/promotions, etc) >>>> in order to get this kite flying. >>>> >>>> Best regards, >>>> >>>> Pierre Smits >>>> >>>> *OFBiz Extensions Marketplace* >>>> http://oem.ofbizci.net/oci-2/ >>>> >>> >>> -- >>> Ron Wheeler >>> President >>> Artifact Software Inc >>> email: [hidden email] >>> skype: ronaldmwheeler >>> phone: 866-970-2435, ext 102 >>> >> > > |
Re: Why A Framework Rewrite Is Necessary
|
|
In reply to this post by Jacques Le Roux
Same feeling on Sharan' role.
BTW, perhaps we can add EDI support to OFBiz: Retailer A's ERP (OFBiz) with EDI adaptor <-> EDI Center (Moqui) <-> Brand A's ERP (OFBiz) Retailer B's ERP (OFBiz) with EDI adaptor <-> <-> Brand B's ERP (OFBiz) -----邮件原件----- 发件人: Jacques Le Roux [mailto:[hidden email]] 发送时间: 2015年10月16日 18:34 收件人: [hidden email] 主题: Re: Why A Framework Rewrite Is Necessary By and large, that also sounds wise to me. Like we said in French : "Il ne faut pas mettre la charrue avant les boeufs" (the cart before the horse) But this also does not mean that nothing should be done. Simply we are not in a hurry and should take the time to collect, not the opinions (we have enough), but the ways to go. And we should not depend on projects to grow (easy to say...) I personally think Sharan is doing an excellent work a that, a kinda PM for the OFBiz project. We don't need a benevolent dictator (no misunderstandingDavid, I don't say that to you, I know you rejected the term and you always proved your openness and understanding, for those - if any - who would doubt, OFBiz is here to prove it) but we need a *more coherent team*. That's exactly how I feel Sharan's work, and it's not an easy one... Jacques Le 16/10/2015 09:41, Pierre Smits a écrit : > On Fri, Oct 16, 2015 at 5:31 AM, David E. Jones <[hidden email]> wrote: > >> >>> On 15 Oct 2015, at 07:58, Adrian Crum < >> [hidden email]> wrote: >>> Keep in mind that much of David's code in OFBiz has been rewritten. So >> yes, we CAN do a better job than him. >> >> I think there’s a name for this logical fallacy… >> >> And this could also be called a logical fallacy. But let us not make it > into a pissing contest again, like we had in the past regarding the > opposing viewpoints on this. > > >>> Also keep in mind that Moqui duplicates some of the problems I listed - >> so by using Moqui, we keep the same problems instead of fixing them. >> >> Could you be more specific, other than the type conversion stuff you >> mentioned many years ago (which I fully disagree with)? >> >> This is not about who is right or wrong, but where the community wants to > go. > > I understand the reluctance of the community, because the impact will be > huge. When looking at the data in OpenHub I see OFBiz having an estimate > effort spend of 519 person years vs 6 for the combined > Moqui-Mantle-HiveMindPM-PopCommerce suite. And one of the reasons behind it > is simple: Many more have worked on OFBiz (from day 1) than on the Moqui > suite. One could even argue that both directions took the same number of > years in duration to get where they are now. Without all the experiences > regarding the OFBiz product there couldn't have been an evolution called > the Moqui suite. > > Coming back to opting for a new direction, as Scott has stated we can have > this in a separate code repository (subproject, like many other Apache > projects do their work) even combined with a new JIRA an Wiki under the > umbrella of the OFBiz project. Based on the comments provided, this seems > like a logical choice to ensure that adopters of the current solution will > keep the support of the community while at the same time ensuring > containment of the new approach. > > But these are mere technical, supportive aspects. The more important issue > is what this new solution will encompass. There are talks of a rewrite, > which sounds like reinvention of the wheel. But I guess it is not like > that. Yet, taking decisions based on a few one-pagers (e.g. > http://www.sandglass-software.com/products/sandglass/documents/Foundation_Brochure.pdf) > are seldom done. Maybe it works for a single person, but I doubt it will > make a community fly. > > Whether fix or rewrite, choices will be made regarding the supporting 3rd > party libraries/solutions and the community needs to understand the impact > to get behind it. So before we embark on the coding trip, let us get into > trying to define a bit more what the new solution will encompass and get > consensus on that. > > Another issue regarding getting the community behind behind this new effort > is this: 'restrict access to the new code'. I guess this meant restrict > write access. Though understandable from a avoidance of dilution/scope > creep point of view, I see this as a wrong direction. This 'proposed' > exclusion of contributors of the kind will bring us back to where this > project came from: discrimination and favouritism. I even doubt that this > is possible under the current principles of the ASF. > Given that this is an enormous undertaking, we need to get as many on board > as possible. Not only to ensure that the decisions (at each level) will get > consensus, but also the ensure that every aspect down the line will get > addressed (e.g. documentation, test definitions, marketing/promotions, etc) > in order to get this kite flying. > > Best regards, > > Pierre Smits > > *OFBiz Extensions Marketplace* > http://oem.ofbizci.net/oci-2/ > |
«
Return to OFBiz - Dev
|
1 view|%1 views
| Free forum by Nabble | Edit this page |