Proposal to modify the testing framework for OFBiz

[taher]

Hello Everyone,

In reference to this thread and the Jira OFBIZ-7254, I'm very happy to
announce that OFBiz is now ready for applying unit tests with only 8 new
lines of code in the build script (r1753143) :)

I recommend we do the following moving forward:

1- Introduce unit tests as much as we can to all components
2- Migrate most of the integration tests we currently have to unit tests
(since they are designed to do very little integration).

This is a great chance for us to practice real TDD in OFBiz

Cheers,

Taher Alkhateeb

On Mon, Jun 13, 2016 at 7:43 AM, Pranay Pandey <
[hidden email]> wrote:

> +1
>
> Best regards,
>
> Pranay Pandey
> HotWax Systems
> http://www.hotwaxsystems.com/
>
> On Fri, Jun 10, 2016 at 7:46 PM, Taher Alkhateeb <
> [hidden email]
> > wrote:
>
> > Hello Everyone,
> >
> > I was able to get a few tests running and this is very doable. But I
> faced
> > a big problem in designing the testing framework because of ANT.
> >
> > The problem
> > ----------------
> > The way the build scripts are designed in OFBiz are very complex. A
> master
> > file calls other files which call other files. And in the middle you have
> > external libraries (ant-contrib) and macros, and variables, and class
> path
> > declarations, and and and ....
> >
> > I cannot declare the tests programmatically (with JUnit test suites)
> > because this means lower level components would depend on higher level
> > components. So I have to do it in ANT, by navigating this maze of build
> > scripts, and it was a headache for me just to read them, let alone modify
> > them to create a testing framework.
> >
> > Suggested Solution
> > ------------------------
> > I suggest to implement the testing framework in Gradle, and simply call
> it
> > from within ant. This is a middle solution that sustains ant for now, but
> > can allow us to switch out later.
> >
> > This means I will just add one more file called build.gradle in the top
> > level directory, and figure out the business logic for calling the test
> > suites from that file
> >
> > I look forward to your feedback.
> >
> > Regards,
> >
> > Taher Alkhateeb
> >
> > On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <
> > [hidden email]>
> > wrote:
> >
> > > Hi Everyone,
> > >
> > > Thank you all for your support, JIRA created in
> > > https://issues.apache.org/jira/browse/OFBIZ-7254
> > >
> > > I will start working on it and try to implement ASAP to get my focus
> back
> > > on refactoring.
> > >
> > > Cheers!
> > >
> > > Taher Alkhateeb
> > >
> > > On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
> > > [hidden email]> wrote:
> > >
> > >> +1
> > >>
> > >> Thanks & Regards
> > >> --
> > >> Deepak Dixit
> > >> www.hotwaxsystems.com
> > >>
> > >> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
> > >> [hidden email]> wrote:
> > >>
> > >> > +1
> > >> >
> > >> > Makes perfect sense.
> > >> >
> > >> > --
> > >> > Thanks & Regards,
> > >> > Mridul Pathak
> > >> > Senior Manager
> > >> > HotWax Systems
> > >> > http://www.hotwaxsystems.com
> > >> >
> > >> > > On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
> > >> [hidden email]>
> > >> > wrote:
> > >> > >
> > >> > > Hello Everyone,
> > >> > >
> > >> > > After refactoring the start component and while starting on the
> base
> > >> > > component I realized that the testing framework for OFBiz is not
> > good.
> > >> > You
> > >> > > cannot do real test driven development or red-green-refactor with
> > the
> > >> > > current setup, hence my proposal to change it. I explain below:
> > >> > >
> > >> > > Problem with current design
> > >> > > ----------------------------------------
> > >> > > - What we have right now is not unit tests, it's really
> integration
> > >> > tests.
> > >> > > You have to start the framework, the database, the service engine,
> > the
> > >> > > entity engine and pretty much everything.
> > >> > > - Testing is very slow, because it's an integration test as I
> > >> mentioned
> > >> > > above. 10 minutes on a good computer!
> > >> > > - There is zero mocking! We actually have to --load-data for
> things
> > to
> > >> > > work. Again, these are integration tests.
> > >> > > - Too complex: Integration tests by their nature are grabbing too
> > >> much.
> > >> > > Mind you, I am not objecting to integration tests (I actually like
> > >> them)
> > >> > > but I am objecting to not having real unit-tests. Unit tests
> should
> > >> all
> > >> > run
> > >> > > in a few seconds.
> > >> > >
> > >> > > Proposed solution
> > >> > > --------------------------
> > >> > > - We keep what is considered real integration tests the way they
> are
> > >> > right
> > >> > > now and keep using them
> > >> > > - We move what should be unit tests into simple JUnit classes, and
> > we
> > >> do
> > >> > > not run them using java -jar ofbiz.jar --test, but instead run
> them
> > >> > > directly from the build.xml script, so these files are not
> > identified
> > >> in
> > >> > > any XML document, but are simply called immediately from the build
> > >> > scripts.
> > >> > > - We clearly mark the difference between integration tests and
> unit
> > >> tests
> > >> > > (inside the source files or in the suite declarations).
> > >> > > - We change the run-tests target in build.xml to run both unit
> tests
> > >> and
> > >> > > integration tests.
> > >> > >
> > >> > > I intend to heavily refactor the framework and I would feel better
> > >> about
> > >> > > introducing this change while refactoring. What do you guys think?
> > >> Ideas?
> > >> > > Suggestions? Approvals and thumbs up?
> > >> > >
> > >> > > Regards,
> > >> > >
> > >> > > Taher Alkhateeb
> > >> >
> > >> >
> > >>
> > >
> > >
> >
>

[Akash Jain]

Thanks Taher for nice initiative!

We are planning to written unit tests to all components under OFBIZ-1463

Thanks and Regards
--
Akash Jain

On Mon, Jul 18, 2016 at 10:36 AM, Taher Alkhateeb <
[hidden email]> wrote:

> Hello Everyone,
>
> In reference to this thread and the Jira OFBIZ-7254, I'm very happy to
> announce that OFBiz is now ready for applying unit tests with only 8 new
> lines of code in the build script (r1753143) :)
>
> I recommend we do the following moving forward:
>
> 1- Introduce unit tests as much as we can to all components
> 2- Migrate most of the integration tests we currently have to unit tests
> (since they are designed to do very little integration).
>
> This is a great chance for us to practice real TDD in OFBiz
>
> Cheers,
>
> Taher Alkhateeb
>
> On Mon, Jun 13, 2016 at 7:43 AM, Pranay Pandey <
> [hidden email]> wrote:
>
> > +1
> >
> > Best regards,
> >
> > Pranay Pandey
> > HotWax Systems
> > http://www.hotwaxsystems.com/
> >
> > On Fri, Jun 10, 2016 at 7:46 PM, Taher Alkhateeb <
> > [hidden email]
> > > wrote:
> >
> > > Hello Everyone,
> > >
> > > I was able to get a few tests running and this is very doable. But I
> > faced
> > > a big problem in designing the testing framework because of ANT.
> > >
> > > The problem
> > > ----------------
> > > The way the build scripts are designed in OFBiz are very complex. A
> > master
> > > file calls other files which call other files. And in the middle you
> have
> > > external libraries (ant-contrib) and macros, and variables, and class
> > path
> > > declarations, and and and ....
> > >
> > > I cannot declare the tests programmatically (with JUnit test suites)
> > > because this means lower level components would depend on higher level
> > > components. So I have to do it in ANT, by navigating this maze of build
> > > scripts, and it was a headache for me just to read them, let alone
> modify
> > > them to create a testing framework.
> > >
> > > Suggested Solution
> > > ------------------------
> > > I suggest to implement the testing framework in Gradle, and simply call
> > it
> > > from within ant. This is a middle solution that sustains ant for now,
> but
> > > can allow us to switch out later.
> > >
> > > This means I will just add one more file called build.gradle in the top
> > > level directory, and figure out the business logic for calling the test
> > > suites from that file
> > >
> > > I look forward to your feedback.
> > >
> > > Regards,
> > >
> > > Taher Alkhateeb
> > >
> > > On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <
> > > [hidden email]>
> > > wrote:
> > >
> > > > Hi Everyone,
> > > >
> > > > Thank you all for your support, JIRA created in
> > > > https://issues.apache.org/jira/browse/OFBIZ-7254
> > > >
> > > > I will start working on it and try to implement ASAP to get my focus
> > back
> > > > on refactoring.
> > > >
> > > > Cheers!
> > > >
> > > > Taher Alkhateeb
> > > >
> > > > On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
> > > > [hidden email]> wrote:
> > > >
> > > >> +1
> > > >>
> > > >> Thanks & Regards
> > > >> --
> > > >> Deepak Dixit
> > > >> www.hotwaxsystems.com
> > > >>
> > > >> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
> > > >> [hidden email]> wrote:
> > > >>
> > > >> > +1
> > > >> >
> > > >> > Makes perfect sense.
> > > >> >
> > > >> > --
> > > >> > Thanks & Regards,
> > > >> > Mridul Pathak
> > > >> > Senior Manager
> > > >> > HotWax Systems
> > > >> > http://www.hotwaxsystems.com
> > > >> >
> > > >> > > On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
> > > >> [hidden email]>
> > > >> > wrote:
> > > >> > >
> > > >> > > Hello Everyone,
> > > >> > >
> > > >> > > After refactoring the start component and while starting on the
> > base
> > > >> > > component I realized that the testing framework for OFBiz is not
> > > good.
> > > >> > You
> > > >> > > cannot do real test driven development or red-green-refactor
> with
> > > the
> > > >> > > current setup, hence my proposal to change it. I explain below:
> > > >> > >
> > > >> > > Problem with current design
> > > >> > > ----------------------------------------
> > > >> > > - What we have right now is not unit tests, it's really
> > integration
> > > >> > tests.
> > > >> > > You have to start the framework, the database, the service
> engine,
> > > the
> > > >> > > entity engine and pretty much everything.
> > > >> > > - Testing is very slow, because it's an integration test as I
> > > >> mentioned
> > > >> > > above. 10 minutes on a good computer!
> > > >> > > - There is zero mocking! We actually have to --load-data for
> > things
> > > to
> > > >> > > work. Again, these are integration tests.
> > > >> > > - Too complex: Integration tests by their nature are grabbing
> too
> > > >> much.
> > > >> > > Mind you, I am not objecting to integration tests (I actually
> like
> > > >> them)
> > > >> > > but I am objecting to not having real unit-tests. Unit tests
> > should
> > > >> all
> > > >> > run
> > > >> > > in a few seconds.
> > > >> > >
> > > >> > > Proposed solution
> > > >> > > --------------------------
> > > >> > > - We keep what is considered real integration tests the way they
> > are
> > > >> > right
> > > >> > > now and keep using them
> > > >> > > - We move what should be unit tests into simple JUnit classes,
> and
> > > we
> > > >> do
> > > >> > > not run them using java -jar ofbiz.jar --test, but instead run
> > them
> > > >> > > directly from the build.xml script, so these files are not
> > > identified
> > > >> in
> > > >> > > any XML document, but are simply called immediately from the
> build
> > > >> > scripts.
> > > >> > > - We clearly mark the difference between integration tests and
> > unit
> > > >> tests
> > > >> > > (inside the source files or in the suite declarations).
> > > >> > > - We change the run-tests target in build.xml to run both unit
> > tests
> > > >> and
> > > >> > > integration tests.
> > > >> > >
> > > >> > > I intend to heavily refactor the framework and I would feel
> better
> > > >> about
> > > >> > > introducing this change while refactoring. What do you guys
> think?
> > > >> Ideas?
> > > >> > > Suggestions? Approvals and thumbs up?
> > > >> > >
> > > >> > > Regards,
> > > >> > >
> > > >> > > Taher Alkhateeb
> > > >> >
> > > >> >
> > > >>
> > > >
> > > >
> > >
> >
>

[taher]

Hello Akash,

Fantastic, I have a few unit tests almost done to be included in the start
component. I will create a new subtask under OFBIZ-1463 to commit the tests
so you can use them as a reference if you like to.

I also recommend that you follow the same directory structure between the
test code and production code. So for example:

Production code: framework/start/src/main/java/org/apache/ofbiz/base/start
Test code: framework/start/src/test/java/org/apache/ofbiz/base/start

The benefit of this hierarchy is that you can access non-public (package
protected) methods for testing. This is in fact exactly what I needed to be
able to apply some of the tests.

Cheers,

Taher Alkhateeb

On Mon, Jul 18, 2016 at 9:22 AM, Akash Jain <[hidden email]>
wrote:

> Thanks Taher for nice initiative!
>
> We are planning to written unit tests to all components under OFBIZ-1463
>
> Thanks and Regards
> --
> Akash Jain
>
> On Mon, Jul 18, 2016 at 10:36 AM, Taher Alkhateeb <
> [hidden email]> wrote:
>
> > Hello Everyone,
> >
> > In reference to this thread and the Jira OFBIZ-7254, I'm very happy to
> > announce that OFBiz is now ready for applying unit tests with only 8 new
> > lines of code in the build script (r1753143) :)
> >
> > I recommend we do the following moving forward:
> >
> > 1- Introduce unit tests as much as we can to all components
> > 2- Migrate most of the integration tests we currently have to unit tests
> > (since they are designed to do very little integration).
> >
> > This is a great chance for us to practice real TDD in OFBiz
> >
> > Cheers,
> >
> > Taher Alkhateeb
> >
> > On Mon, Jun 13, 2016 at 7:43 AM, Pranay Pandey <
> > [hidden email]> wrote:
> >
> > > +1
> > >
> > > Best regards,
> > >
> > > Pranay Pandey
> > > HotWax Systems
> > > http://www.hotwaxsystems.com/
> > >
> > > On Fri, Jun 10, 2016 at 7:46 PM, Taher Alkhateeb <
> > > [hidden email]
> > > > wrote:
> > >
> > > > Hello Everyone,
> > > >
> > > > I was able to get a few tests running and this is very doable. But I
> > > faced
> > > > a big problem in designing the testing framework because of ANT.
> > > >
> > > > The problem
> > > > ----------------
> > > > The way the build scripts are designed in OFBiz are very complex. A
> > > master
> > > > file calls other files which call other files. And in the middle you
> > have
> > > > external libraries (ant-contrib) and macros, and variables, and class
> > > path
> > > > declarations, and and and ....
> > > >
> > > > I cannot declare the tests programmatically (with JUnit test suites)
> > > > because this means lower level components would depend on higher
> level
> > > > components. So I have to do it in ANT, by navigating this maze of
> build
> > > > scripts, and it was a headache for me just to read them, let alone
> > modify
> > > > them to create a testing framework.
> > > >
> > > > Suggested Solution
> > > > ------------------------
> > > > I suggest to implement the testing framework in Gradle, and simply
> call
> > > it
> > > > from within ant. This is a middle solution that sustains ant for now,
> > but
> > > > can allow us to switch out later.
> > > >
> > > > This means I will just add one more file called build.gradle in the
> top
> > > > level directory, and figure out the business logic for calling the
> test
> > > > suites from that file
> > > >
> > > > I look forward to your feedback.
> > > >
> > > > Regards,
> > > >
> > > > Taher Alkhateeb
> > > >
> > > > On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <
> > > > [hidden email]>
> > > > wrote:
> > > >
> > > > > Hi Everyone,
> > > > >
> > > > > Thank you all for your support, JIRA created in
> > > > > https://issues.apache.org/jira/browse/OFBIZ-7254
> > > > >
> > > > > I will start working on it and try to implement ASAP to get my
> focus
> > > back
> > > > > on refactoring.
> > > > >
> > > > > Cheers!
> > > > >
> > > > > Taher Alkhateeb
> > > > >
> > > > > On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
> > > > > [hidden email]> wrote:
> > > > >
> > > > >> +1
> > > > >>
> > > > >> Thanks & Regards
> > > > >> --
> > > > >> Deepak Dixit
> > > > >> www.hotwaxsystems.com
> > > > >>
> > > > >> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
> > > > >> [hidden email]> wrote:
> > > > >>
> > > > >> > +1
> > > > >> >
> > > > >> > Makes perfect sense.
> > > > >> >
> > > > >> > --
> > > > >> > Thanks & Regards,
> > > > >> > Mridul Pathak
> > > > >> > Senior Manager
> > > > >> > HotWax Systems
> > > > >> > http://www.hotwaxsystems.com
> > > > >> >
> > > > >> > > On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
> > > > >> [hidden email]>
> > > > >> > wrote:
> > > > >> > >
> > > > >> > > Hello Everyone,
> > > > >> > >
> > > > >> > > After refactoring the start component and while starting on
> the
> > > base
> > > > >> > > component I realized that the testing framework for OFBiz is
> not
> > > > good.
> > > > >> > You
> > > > >> > > cannot do real test driven development or red-green-refactor
> > with
> > > > the
> > > > >> > > current setup, hence my proposal to change it. I explain
> below:
> > > > >> > >
> > > > >> > > Problem with current design
> > > > >> > > ----------------------------------------
> > > > >> > > - What we have right now is not unit tests, it's really
> > > integration
> > > > >> > tests.
> > > > >> > > You have to start the framework, the database, the service
> > engine,
> > > > the
> > > > >> > > entity engine and pretty much everything.
> > > > >> > > - Testing is very slow, because it's an integration test as I
> > > > >> mentioned
> > > > >> > > above. 10 minutes on a good computer!
> > > > >> > > - There is zero mocking! We actually have to --load-data for
> > > things
> > > > to
> > > > >> > > work. Again, these are integration tests.
> > > > >> > > - Too complex: Integration tests by their nature are grabbing
> > too
> > > > >> much.
> > > > >> > > Mind you, I am not objecting to integration tests (I actually
> > like
> > > > >> them)
> > > > >> > > but I am objecting to not having real unit-tests. Unit tests
> > > should
> > > > >> all
> > > > >> > run
> > > > >> > > in a few seconds.
> > > > >> > >
> > > > >> > > Proposed solution
> > > > >> > > --------------------------
> > > > >> > > - We keep what is considered real integration tests the way
> they
> > > are
> > > > >> > right
> > > > >> > > now and keep using them
> > > > >> > > - We move what should be unit tests into simple JUnit classes,
> > and
> > > > we
> > > > >> do
> > > > >> > > not run them using java -jar ofbiz.jar --test, but instead run
> > > them
> > > > >> > > directly from the build.xml script, so these files are not
> > > > identified
> > > > >> in
> > > > >> > > any XML document, but are simply called immediately from the
> > build
> > > > >> > scripts.
> > > > >> > > - We clearly mark the difference between integration tests and
> > > unit
> > > > >> tests
> > > > >> > > (inside the source files or in the suite declarations).
> > > > >> > > - We change the run-tests target in build.xml to run both unit
> > > tests
> > > > >> and
> > > > >> > > integration tests.
> > > > >> > >
> > > > >> > > I intend to heavily refactor the framework and I would feel
> > better
> > > > >> about
> > > > >> > > introducing this change while refactoring. What do you guys
> > think?
> > > > >> Ideas?
> > > > >> > > Suggestions? Approvals and thumbs up?
> > > > >> > >
> > > > >> > > Regards,
> > > > >> > >
> > > > >> > > Taher Alkhateeb
> > > > >> >
> > > > >> >
> > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
>

[Jacques Le Roux]

It's very cool to see this old Jira revived :)

Jacques


Le 18/07/2016 à 08:22, Akash Jain a écrit :

> Thanks Taher for nice initiative!
>
> We are planning to written unit tests to all components under OFBIZ-1463
>
> Thanks and Regards
> --
> Akash Jain
>
> On Mon, Jul 18, 2016 at 10:36 AM, Taher Alkhateeb <
> [hidden email]> wrote:
>
>> Hello Everyone,
>>
>> In reference to this thread and the Jira OFBIZ-7254, I'm very happy to
>> announce that OFBiz is now ready for applying unit tests with only 8 new
>> lines of code in the build script (r1753143) :)
>>
>> I recommend we do the following moving forward:
>>
>> 1- Introduce unit tests as much as we can to all components
>> 2- Migrate most of the integration tests we currently have to unit tests
>> (since they are designed to do very little integration).
>>
>> This is a great chance for us to practice real TDD in OFBiz
>>
>> Cheers,
>>
>> Taher Alkhateeb
>>
>> On Mon, Jun 13, 2016 at 7:43 AM, Pranay Pandey <
>> [hidden email]> wrote:
>>
>>> +1
>>>
>>> Best regards,
>>>
>>> Pranay Pandey
>>> HotWax Systems
>>> http://www.hotwaxsystems.com/
>>>
>>> On Fri, Jun 10, 2016 at 7:46 PM, Taher Alkhateeb <
>>> [hidden email]
>>>> wrote:
>>>> Hello Everyone,
>>>>
>>>> I was able to get a few tests running and this is very doable. But I
>>> faced
>>>> a big problem in designing the testing framework because of ANT.
>>>>
>>>> The problem
>>>> ----------------
>>>> The way the build scripts are designed in OFBiz are very complex. A
>>> master
>>>> file calls other files which call other files. And in the middle you
>> have
>>>> external libraries (ant-contrib) and macros, and variables, and class
>>> path
>>>> declarations, and and and ....
>>>>
>>>> I cannot declare the tests programmatically (with JUnit test suites)
>>>> because this means lower level components would depend on higher level
>>>> components. So I have to do it in ANT, by navigating this maze of build
>>>> scripts, and it was a headache for me just to read them, let alone
>> modify
>>>> them to create a testing framework.
>>>>
>>>> Suggested Solution
>>>> ------------------------
>>>> I suggest to implement the testing framework in Gradle, and simply call
>>> it
>>>> from within ant. This is a middle solution that sustains ant for now,
>> but
>>>> can allow us to switch out later.
>>>>
>>>> This means I will just add one more file called build.gradle in the top
>>>> level directory, and figure out the business logic for calling the test
>>>> suites from that file
>>>>
>>>> I look forward to your feedback.
>>>>
>>>> Regards,
>>>>
>>>> Taher Alkhateeb
>>>>
>>>> On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <
>>>> [hidden email]>
>>>> wrote:
>>>>
>>>>> Hi Everyone,
>>>>>
>>>>> Thank you all for your support, JIRA created in
>>>>> https://issues.apache.org/jira/browse/OFBIZ-7254
>>>>>
>>>>> I will start working on it and try to implement ASAP to get my focus
>>> back
>>>>> on refactoring.
>>>>>
>>>>> Cheers!
>>>>>
>>>>> Taher Alkhateeb
>>>>>
>>>>> On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
>>>>> [hidden email]> wrote:
>>>>>
>>>>>> +1
>>>>>>
>>>>>> Thanks & Regards
>>>>>> --
>>>>>> Deepak Dixit
>>>>>> www.hotwaxsystems.com
>>>>>>
>>>>>> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
>>>>>> [hidden email]> wrote:
>>>>>>
>>>>>>> +1
>>>>>>>
>>>>>>> Makes perfect sense.
>>>>>>>
>>>>>>> --
>>>>>>> Thanks & Regards,
>>>>>>> Mridul Pathak
>>>>>>> Senior Manager
>>>>>>> HotWax Systems
>>>>>>> http://www.hotwaxsystems.com
>>>>>>>
>>>>>>>> On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
>>>>>> [hidden email]>
>>>>>>> wrote:
>>>>>>>> Hello Everyone,
>>>>>>>>
>>>>>>>> After refactoring the start component and while starting on the
>>> base
>>>>>>>> component I realized that the testing framework for OFBiz is not
>>>> good.
>>>>>>> You
>>>>>>>> cannot do real test driven development or red-green-refactor
>> with
>>>> the
>>>>>>>> current setup, hence my proposal to change it. I explain below:
>>>>>>>>
>>>>>>>> Problem with current design
>>>>>>>> ----------------------------------------
>>>>>>>> - What we have right now is not unit tests, it's really
>>> integration
>>>>>>> tests.
>>>>>>>> You have to start the framework, the database, the service
>> engine,
>>>> the
>>>>>>>> entity engine and pretty much everything.
>>>>>>>> - Testing is very slow, because it's an integration test as I
>>>>>> mentioned
>>>>>>>> above. 10 minutes on a good computer!
>>>>>>>> - There is zero mocking! We actually have to --load-data for
>>> things
>>>> to
>>>>>>>> work. Again, these are integration tests.
>>>>>>>> - Too complex: Integration tests by their nature are grabbing
>> too
>>>>>> much.
>>>>>>>> Mind you, I am not objecting to integration tests (I actually
>> like
>>>>>> them)
>>>>>>>> but I am objecting to not having real unit-tests. Unit tests
>>> should
>>>>>> all
>>>>>>> run
>>>>>>>> in a few seconds.
>>>>>>>>
>>>>>>>> Proposed solution
>>>>>>>> --------------------------
>>>>>>>> - We keep what is considered real integration tests the way they
>>> are
>>>>>>> right
>>>>>>>> now and keep using them
>>>>>>>> - We move what should be unit tests into simple JUnit classes,
>> and
>>>> we
>>>>>> do
>>>>>>>> not run them using java -jar ofbiz.jar --test, but instead run
>>> them
>>>>>>>> directly from the build.xml script, so these files are not
>>>> identified
>>>>>> in
>>>>>>>> any XML document, but are simply called immediately from the
>> build
>>>>>>> scripts.
>>>>>>>> - We clearly mark the difference between integration tests and
>>> unit
>>>>>> tests
>>>>>>>> (inside the source files or in the suite declarations).
>>>>>>>> - We change the run-tests target in build.xml to run both unit
>>> tests
>>>>>> and
>>>>>>>> integration tests.
>>>>>>>>
>>>>>>>> I intend to heavily refactor the framework and I would feel
>> better
>>>>>> about
>>>>>>>> introducing this change while refactoring. What do you guys
>> think?
>>>>>> Ideas?
>>>>>>>> Suggestions? Approvals and thumbs up?
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>>
>>>>>>>> Taher Alkhateeb
>>>>>>>
>>>>>

[Hans Bakker]

great initiative Taher, finally someone who values automated tests to
enable TDD but also leading to continuous improvement and finally
continuous deployment....

Regards,
Hans Bakker
CEO AntWebsystems.com

On 18/07/16 12:06, Taher Alkhateeb wrote:

> Hello Everyone,
>
> In reference to this thread and the Jira OFBIZ-7254, I'm very happy to
> announce that OFBiz is now ready for applying unit tests with only 8 new
> lines of code in the build script (r1753143) :)
>
> I recommend we do the following moving forward:
>
> 1- Introduce unit tests as much as we can to all components
> 2- Migrate most of the integration tests we currently have to unit tests
> (since they are designed to do very little integration).
>
> This is a great chance for us to practice real TDD in OFBiz
>
> Cheers,
>
> Taher Alkhateeb
>
> On Mon, Jun 13, 2016 at 7:43 AM, Pranay Pandey <
> [hidden email]> wrote:
>
>> +1
>>
>> Best regards,
>>
>> Pranay Pandey
>> HotWax Systems
>> http://www.hotwaxsystems.com/
>>
>> On Fri, Jun 10, 2016 at 7:46 PM, Taher Alkhateeb <
>> [hidden email]
>>> wrote:
>>
>>> Hello Everyone,
>>>
>>> I was able to get a few tests running and this is very doable. But I
>> faced
>>> a big problem in designing the testing framework because of ANT.
>>>
>>> The problem
>>> ----------------
>>> The way the build scripts are designed in OFBiz are very complex. A
>> master
>>> file calls other files which call other files. And in the middle you have
>>> external libraries (ant-contrib) and macros, and variables, and class
>> path
>>> declarations, and and and ....
>>>
>>> I cannot declare the tests programmatically (with JUnit test suites)
>>> because this means lower level components would depend on higher level
>>> components. So I have to do it in ANT, by navigating this maze of build
>>> scripts, and it was a headache for me just to read them, let alone modify
>>> them to create a testing framework.
>>>
>>> Suggested Solution
>>> ------------------------
>>> I suggest to implement the testing framework in Gradle, and simply call
>> it
>>> from within ant. This is a middle solution that sustains ant for now, but
>>> can allow us to switch out later.
>>>
>>> This means I will just add one more file called build.gradle in the top
>>> level directory, and figure out the business logic for calling the test
>>> suites from that file
>>>
>>> I look forward to your feedback.
>>>
>>> Regards,
>>>
>>> Taher Alkhateeb
>>>
>>> On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <
>>> [hidden email]>
>>> wrote:
>>>
>>>> Hi Everyone,
>>>>
>>>> Thank you all for your support, JIRA created in
>>>> https://issues.apache.org/jira/browse/OFBIZ-7254
>>>>
>>>> I will start working on it and try to implement ASAP to get my focus
>> back
>>>> on refactoring.
>>>>
>>>> Cheers!
>>>>
>>>> Taher Alkhateeb
>>>>
>>>> On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
>>>> [hidden email]> wrote:
>>>>
>>>>> +1
>>>>>
>>>>> Thanks & Regards
>>>>> --
>>>>> Deepak Dixit
>>>>> www.hotwaxsystems.com
>>>>>
>>>>> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
>>>>> [hidden email]> wrote:
>>>>>
>>>>>> +1
>>>>>>
>>>>>> Makes perfect sense.
>>>>>>
>>>>>> --
>>>>>> Thanks & Regards,
>>>>>> Mridul Pathak
>>>>>> Senior Manager
>>>>>> HotWax Systems
>>>>>> http://www.hotwaxsystems.com
>>>>>>
>>>>>>> On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
>>>>> [hidden email]>
>>>>>> wrote:
>>>>>>>
>>>>>>> Hello Everyone,
>>>>>>>
>>>>>>> After refactoring the start component and while starting on the
>> base
>>>>>>> component I realized that the testing framework for OFBiz is not
>>> good.
>>>>>> You
>>>>>>> cannot do real test driven development or red-green-refactor with
>>> the
>>>>>>> current setup, hence my proposal to change it. I explain below:
>>>>>>>
>>>>>>> Problem with current design
>>>>>>> ----------------------------------------
>>>>>>> - What we have right now is not unit tests, it's really
>> integration
>>>>>> tests.
>>>>>>> You have to start the framework, the database, the service engine,
>>> the
>>>>>>> entity engine and pretty much everything.
>>>>>>> - Testing is very slow, because it's an integration test as I
>>>>> mentioned
>>>>>>> above. 10 minutes on a good computer!
>>>>>>> - There is zero mocking! We actually have to --load-data for
>> things
>>> to
>>>>>>> work. Again, these are integration tests.
>>>>>>> - Too complex: Integration tests by their nature are grabbing too
>>>>> much.
>>>>>>> Mind you, I am not objecting to integration tests (I actually like
>>>>> them)
>>>>>>> but I am objecting to not having real unit-tests. Unit tests
>> should
>>>>> all
>>>>>> run
>>>>>>> in a few seconds.
>>>>>>>
>>>>>>> Proposed solution
>>>>>>> --------------------------
>>>>>>> - We keep what is considered real integration tests the way they
>> are
>>>>>> right
>>>>>>> now and keep using them
>>>>>>> - We move what should be unit tests into simple JUnit classes, and
>>> we
>>>>> do
>>>>>>> not run them using java -jar ofbiz.jar --test, but instead run
>> them
>>>>>>> directly from the build.xml script, so these files are not
>>> identified
>>>>> in
>>>>>>> any XML document, but are simply called immediately from the build
>>>>>> scripts.
>>>>>>> - We clearly mark the difference between integration tests and
>> unit
>>>>> tests
>>>>>>> (inside the source files or in the suite declarations).
>>>>>>> - We change the run-tests target in build.xml to run both unit
>> tests
>>>>> and
>>>>>>> integration tests.
>>>>>>>
>>>>>>> I intend to heavily refactor the framework and I would feel better
>>>>> about
>>>>>>> introducing this change while refactoring. What do you guys think?
>>>>> Ideas?
>>>>>>> Suggestions? Approvals and thumbs up?
>>>>>>>
>>>>>>> Regards,
>>>>>>>
>>>>>>> Taher Alkhateeb
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>

--

Regards,

Hans Bakker
CEO, http://antwebsystems.com

[Scott Gray-3]

I know I'm late to the party here, but I just want to say that I think
integration tests have far greater value to OFBiz than unit tests.  Mostly
because we tend to have quite a low number of tests and integration tests
give us much better coverage per line of test code and the tests are much
closer to the real world scenarios the application might encounter.

I don't really see how unit tests could be applied to non-framework testing
in a useful manner, could you expand on your vision in that regard?  I mean
would we be testing something smaller than a service as the 'unit'?  What
would we mock? Would transaction management still be active? What happens
when the service calls another service, I guess we mock the response from
that service (how)?

It just seems a very complicated method to achieve a less thorough but
albeit faster (maybe) test result.

A build, data load and full test run takes 4m 9s on my laptop.  Excluding
build, data load and framework tests: the application level tests take 35s,
not very expensive IMO.  The data load time can be reduced to practically
nothing by copying a clean slate database into runtime for each run.

I'm mostly just suggesting we be wary of adding complicated testing
procedures in the hope of achieving some 'best practice' result which in
reality will provide minimal benefits.

Regards
Scott

On 18 July 2016 at 18:57, Taher Alkhateeb <[hidden email]>
wrote:

> Hello Akash,
>
> Fantastic, I have a few unit tests almost done to be included in the start
> component. I will create a new subtask under OFBIZ-1463 to commit the tests
> so you can use them as a reference if you like to.
>
> I also recommend that you follow the same directory structure between the
> test code and production code. So for example:
>
> Production code: framework/start/src/main/java/org/apache/ofbiz/base/start
> Test code: framework/start/src/test/java/org/apache/ofbiz/base/start
>
> The benefit of this hierarchy is that you can access non-public (package
> protected) methods for testing. This is in fact exactly what I needed to be
> able to apply some of the tests.
>
> Cheers,
>
> Taher Alkhateeb
>
> On Mon, Jul 18, 2016 at 9:22 AM, Akash Jain <[hidden email]>
> wrote:
>
> > Thanks Taher for nice initiative!
> >
> > We are planning to written unit tests to all components under OFBIZ-1463
> >
> > Thanks and Regards
> > --
> > Akash Jain
> >
> > On Mon, Jul 18, 2016 at 10:36 AM, Taher Alkhateeb <
> > [hidden email]> wrote:
> >
> > > Hello Everyone,
> > >
> > > In reference to this thread and the Jira OFBIZ-7254, I'm very happy to
> > > announce that OFBiz is now ready for applying unit tests with only 8
> new
> > > lines of code in the build script (r1753143) :)
> > >
> > > I recommend we do the following moving forward:
> > >
> > > 1- Introduce unit tests as much as we can to all components
> > > 2- Migrate most of the integration tests we currently have to unit
> tests
> > > (since they are designed to do very little integration).
> > >
> > > This is a great chance for us to practice real TDD in OFBiz
> > >
> > > Cheers,
> > >
> > > Taher Alkhateeb
> > >
> > > On Mon, Jun 13, 2016 at 7:43 AM, Pranay Pandey <
> > > [hidden email]> wrote:
> > >
> > > > +1
> > > >
> > > > Best regards,
> > > >
> > > > Pranay Pandey
> > > > HotWax Systems
> > > > http://www.hotwaxsystems.com/
> > > >
> > > > On Fri, Jun 10, 2016 at 7:46 PM, Taher Alkhateeb <
> > > > [hidden email]
> > > > > wrote:
> > > >
> > > > > Hello Everyone,
> > > > >
> > > > > I was able to get a few tests running and this is very doable. But
> I
> > > > faced
> > > > > a big problem in designing the testing framework because of ANT.
> > > > >
> > > > > The problem
> > > > > ----------------
> > > > > The way the build scripts are designed in OFBiz are very complex. A
> > > > master
> > > > > file calls other files which call other files. And in the middle
> you
> > > have
> > > > > external libraries (ant-contrib) and macros, and variables, and
> class
> > > > path
> > > > > declarations, and and and ....
> > > > >
> > > > > I cannot declare the tests programmatically (with JUnit test
> suites)
> > > > > because this means lower level components would depend on higher
> > level
> > > > > components. So I have to do it in ANT, by navigating this maze of
> > build
> > > > > scripts, and it was a headache for me just to read them, let alone
> > > modify
> > > > > them to create a testing framework.
> > > > >
> > > > > Suggested Solution
> > > > > ------------------------
> > > > > I suggest to implement the testing framework in Gradle, and simply
> > call
> > > > it
> > > > > from within ant. This is a middle solution that sustains ant for
> now,
> > > but
> > > > > can allow us to switch out later.
> > > > >
> > > > > This means I will just add one more file called build.gradle in the
> > top
> > > > > level directory, and figure out the business logic for calling the
> > test
> > > > > suites from that file
> > > > >
> > > > > I look forward to your feedback.
> > > > >
> > > > > Regards,
> > > > >
> > > > > Taher Alkhateeb
> > > > >
> > > > > On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <
> > > > > [hidden email]>
> > > > > wrote:
> > > > >
> > > > > > Hi Everyone,
> > > > > >
> > > > > > Thank you all for your support, JIRA created in
> > > > > > https://issues.apache.org/jira/browse/OFBIZ-7254
> > > > > >
> > > > > > I will start working on it and try to implement ASAP to get my
> > focus
> > > > back
> > > > > > on refactoring.
> > > > > >
> > > > > > Cheers!
> > > > > >
> > > > > > Taher Alkhateeb
> > > > > >
> > > > > > On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
> > > > > > [hidden email]> wrote:
> > > > > >
> > > > > >> +1
> > > > > >>
> > > > > >> Thanks & Regards
> > > > > >> --
> > > > > >> Deepak Dixit
> > > > > >> www.hotwaxsystems.com
> > > > > >>
> > > > > >> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
> > > > > >> [hidden email]> wrote:
> > > > > >>
> > > > > >> > +1
> > > > > >> >
> > > > > >> > Makes perfect sense.
> > > > > >> >
> > > > > >> > --
> > > > > >> > Thanks & Regards,
> > > > > >> > Mridul Pathak
> > > > > >> > Senior Manager
> > > > > >> > HotWax Systems
> > > > > >> > http://www.hotwaxsystems.com
> > > > > >> >
> > > > > >> > > On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
> > > > > >> [hidden email]>
> > > > > >> > wrote:
> > > > > >> > >
> > > > > >> > > Hello Everyone,
> > > > > >> > >
> > > > > >> > > After refactoring the start component and while starting on
> > the
> > > > base
> > > > > >> > > component I realized that the testing framework for OFBiz is
> > not
> > > > > good.
> > > > > >> > You
> > > > > >> > > cannot do real test driven development or red-green-refactor
> > > with
> > > > > the
> > > > > >> > > current setup, hence my proposal to change it. I explain
> > below:
> > > > > >> > >
> > > > > >> > > Problem with current design
> > > > > >> > > ----------------------------------------
> > > > > >> > > - What we have right now is not unit tests, it's really
> > > > integration
> > > > > >> > tests.
> > > > > >> > > You have to start the framework, the database, the service
> > > engine,
> > > > > the
> > > > > >> > > entity engine and pretty much everything.
> > > > > >> > > - Testing is very slow, because it's an integration test as
> I
> > > > > >> mentioned
> > > > > >> > > above. 10 minutes on a good computer!
> > > > > >> > > - There is zero mocking! We actually have to --load-data for
> > > > things
> > > > > to
> > > > > >> > > work. Again, these are integration tests.
> > > > > >> > > - Too complex: Integration tests by their nature are
> grabbing
> > > too
> > > > > >> much.
> > > > > >> > > Mind you, I am not objecting to integration tests (I
> actually
> > > like
> > > > > >> them)
> > > > > >> > > but I am objecting to not having real unit-tests. Unit tests
> > > > should
> > > > > >> all
> > > > > >> > run
> > > > > >> > > in a few seconds.
> > > > > >> > >
> > > > > >> > > Proposed solution
> > > > > >> > > --------------------------
> > > > > >> > > - We keep what is considered real integration tests the way
> > they
> > > > are
> > > > > >> > right
> > > > > >> > > now and keep using them
> > > > > >> > > - We move what should be unit tests into simple JUnit
> classes,
> > > and
> > > > > we
> > > > > >> do
> > > > > >> > > not run them using java -jar ofbiz.jar --test, but instead
> run
> > > > them
> > > > > >> > > directly from the build.xml script, so these files are not
> > > > > identified
> > > > > >> in
> > > > > >> > > any XML document, but are simply called immediately from the
> > > build
> > > > > >> > scripts.
> > > > > >> > > - We clearly mark the difference between integration tests
> and
> > > > unit
> > > > > >> tests
> > > > > >> > > (inside the source files or in the suite declarations).
> > > > > >> > > - We change the run-tests target in build.xml to run both
> unit
> > > > tests
> > > > > >> and
> > > > > >> > > integration tests.
> > > > > >> > >
> > > > > >> > > I intend to heavily refactor the framework and I would feel
> > > better
> > > > > >> about
> > > > > >> > > introducing this change while refactoring. What do you guys
> > > think?
> > > > > >> Ideas?
> > > > > >> > > Suggestions? Approvals and thumbs up?
> > > > > >> > >
> > > > > >> > > Regards,
> > > > > >> > >
> > > > > >> > > Taher Alkhateeb
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

[Ron Wheeler]

You are probably right that integration testing has a lot more value.
However, unit tests are still useful in their own right and may
encourage people to write smaller classes that can be tested easily.

As you point out, integration testing can be done at several levels with
mock implementations used to limit the scope of the tests.

A full set of integration tests would take a long time to run and even
longer to construct.

You may also be right that building a lot of new unit tests to test old
features may not be very valuable but getting developers to move towards
TDD might reduce the number of regressions.
Unit testing does, at least, give positive proof of what has been tested
when new code is presented for review.
It also demonstrates the use of a class which may add to the specifications.

It is usually possible to exclude tests that are redundant or not
relative to the new features being added.
There is not a lot of point in running low-level unit tests repeatedly
in a development cycle if you are not changing the classes being tested.
This is trivial to manage in Maven with jUnit and Surefire.
Not sure about the Gradle build.

Proper integration tests that cover load testing, synchronization,
caching and locking under tests of all UI features would be wonderful to
have but would take a long time to construct given the number of feature
combinations, stack configurations and use cases.
A full test would take a long time to run and would be something that
would be useful only a few times in each release cycle.
Very useful if versions are going to be released with any sort of
implied warranty about quality.

The current activity is a big step forward and Taher's leadership in
modernizing the framework seems to have inspired a lot of new activity
and a more modern way of thinking about software development.

Well done.


Ron


On 18/07/2016 10:19 PM, Scott Gray wrote:

> I know I'm late to the party here, but I just want to say that I think
> integration tests have far greater value to OFBiz than unit tests.  Mostly
> because we tend to have quite a low number of tests and integration tests
> give us much better coverage per line of test code and the tests are much
> closer to the real world scenarios the application might encounter.
>
> I don't really see how unit tests could be applied to non-framework testing
> in a useful manner, could you expand on your vision in that regard?  I mean
> would we be testing something smaller than a service as the 'unit'?  What
> would we mock? Would transaction management still be active? What happens
> when the service calls another service, I guess we mock the response from
> that service (how)?
>
> It just seems a very complicated method to achieve a less thorough but
> albeit faster (maybe) test result.
>
> A build, data load and full test run takes 4m 9s on my laptop.  Excluding
> build, data load and framework tests: the application level tests take 35s,
> not very expensive IMO.  The data load time can be reduced to practically
> nothing by copying a clean slate database into runtime for each run.
>
> I'm mostly just suggesting we be wary of adding complicated testing
> procedures in the hope of achieving some 'best practice' result which in
> reality will provide minimal benefits.
>
> Regards
> Scott
>
> On 18 July 2016 at 18:57, Taher Alkhateeb <[hidden email]>
> wrote:
>
>> Hello Akash,
>>
>> Fantastic, I have a few unit tests almost done to be included in the start
>> component. I will create a new subtask under OFBIZ-1463 to commit the tests
>> so you can use them as a reference if you like to.
>>
>> I also recommend that you follow the same directory structure between the
>> test code and production code. So for example:
>>
>> Production code: framework/start/src/main/java/org/apache/ofbiz/base/start
>> Test code: framework/start/src/test/java/org/apache/ofbiz/base/start
>>
>> The benefit of this hierarchy is that you can access non-public (package
>> protected) methods for testing. This is in fact exactly what I needed to be
>> able to apply some of the tests.
>>
>> Cheers,
>>
>> Taher Alkhateeb
>>
>> On Mon, Jul 18, 2016 at 9:22 AM, Akash Jain <[hidden email]>
>> wrote:
>>
>>> Thanks Taher for nice initiative!
>>>
>>> We are planning to written unit tests to all components under OFBIZ-1463
>>>
>>> Thanks and Regards
>>> --
>>> Akash Jain
>>>
>>> On Mon, Jul 18, 2016 at 10:36 AM, Taher Alkhateeb <
>>> [hidden email]> wrote:
>>>
>>>> Hello Everyone,
>>>>
>>>> In reference to this thread and the Jira OFBIZ-7254, I'm very happy to
>>>> announce that OFBiz is now ready for applying unit tests with only 8
>> new
>>>> lines of code in the build script (r1753143) :)
>>>>
>>>> I recommend we do the following moving forward:
>>>>
>>>> 1- Introduce unit tests as much as we can to all components
>>>> 2- Migrate most of the integration tests we currently have to unit
>> tests
>>>> (since they are designed to do very little integration).
>>>>
>>>> This is a great chance for us to practice real TDD in OFBiz
>>>>
>>>> Cheers,
>>>>
>>>> Taher Alkhateeb
>>>>
>>>> On Mon, Jun 13, 2016 at 7:43 AM, Pranay Pandey <
>>>> [hidden email]> wrote:
>>>>
>>>>> +1
>>>>>
>>>>> Best regards,
>>>>>
>>>>> Pranay Pandey
>>>>> HotWax Systems
>>>>> http://www.hotwaxsystems.com/
>>>>>
>>>>> On Fri, Jun 10, 2016 at 7:46 PM, Taher Alkhateeb <
>>>>> [hidden email]
>>>>>> wrote:
>>>>>> Hello Everyone,
>>>>>>
>>>>>> I was able to get a few tests running and this is very doable. But
>> I
>>>>> faced
>>>>>> a big problem in designing the testing framework because of ANT.
>>>>>>
>>>>>> The problem
>>>>>> ----------------
>>>>>> The way the build scripts are designed in OFBiz are very complex. A
>>>>> master
>>>>>> file calls other files which call other files. And in the middle
>> you
>>>> have
>>>>>> external libraries (ant-contrib) and macros, and variables, and
>> class
>>>>> path
>>>>>> declarations, and and and ....
>>>>>>
>>>>>> I cannot declare the tests programmatically (with JUnit test
>> suites)
>>>>>> because this means lower level components would depend on higher
>>> level
>>>>>> components. So I have to do it in ANT, by navigating this maze of
>>> build
>>>>>> scripts, and it was a headache for me just to read them, let alone
>>>> modify
>>>>>> them to create a testing framework.
>>>>>>
>>>>>> Suggested Solution
>>>>>> ------------------------
>>>>>> I suggest to implement the testing framework in Gradle, and simply
>>> call
>>>>> it
>>>>>> from within ant. This is a middle solution that sustains ant for
>> now,
>>>> but
>>>>>> can allow us to switch out later.
>>>>>>
>>>>>> This means I will just add one more file called build.gradle in the
>>> top
>>>>>> level directory, and figure out the business logic for calling the
>>> test
>>>>>> suites from that file
>>>>>>
>>>>>> I look forward to your feedback.
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> Taher Alkhateeb
>>>>>>
>>>>>> On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <
>>>>>> [hidden email]>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Everyone,
>>>>>>>
>>>>>>> Thank you all for your support, JIRA created in
>>>>>>> https://issues.apache.org/jira/browse/OFBIZ-7254
>>>>>>>
>>>>>>> I will start working on it and try to implement ASAP to get my
>>> focus
>>>>> back
>>>>>>> on refactoring.
>>>>>>>
>>>>>>> Cheers!
>>>>>>>
>>>>>>> Taher Alkhateeb
>>>>>>>
>>>>>>> On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
>>>>>>> [hidden email]> wrote:
>>>>>>>
>>>>>>>> +1
>>>>>>>>
>>>>>>>> Thanks & Regards
>>>>>>>> --
>>>>>>>> Deepak Dixit
>>>>>>>> www.hotwaxsystems.com
>>>>>>>>
>>>>>>>> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
>>>>>>>> [hidden email]> wrote:
>>>>>>>>
>>>>>>>>> +1
>>>>>>>>>
>>>>>>>>> Makes perfect sense.
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Thanks & Regards,
>>>>>>>>> Mridul Pathak
>>>>>>>>> Senior Manager
>>>>>>>>> HotWax Systems
>>>>>>>>> http://www.hotwaxsystems.com
>>>>>>>>>
>>>>>>>>>> On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
>>>>>>>> [hidden email]>
>>>>>>>>> wrote:
>>>>>>>>>> Hello Everyone,
>>>>>>>>>>
>>>>>>>>>> After refactoring the start component and while starting on
>>> the
>>>>> base
>>>>>>>>>> component I realized that the testing framework for OFBiz is
>>> not
>>>>>> good.
>>>>>>>>> You
>>>>>>>>>> cannot do real test driven development or red-green-refactor
>>>> with
>>>>>> the
>>>>>>>>>> current setup, hence my proposal to change it. I explain
>>> below:
>>>>>>>>>> Problem with current design
>>>>>>>>>> ----------------------------------------
>>>>>>>>>> - What we have right now is not unit tests, it's really
>>>>> integration
>>>>>>>>> tests.
>>>>>>>>>> You have to start the framework, the database, the service
>>>> engine,
>>>>>> the
>>>>>>>>>> entity engine and pretty much everything.
>>>>>>>>>> - Testing is very slow, because it's an integration test as
>> I
>>>>>>>> mentioned
>>>>>>>>>> above. 10 minutes on a good computer!
>>>>>>>>>> - There is zero mocking! We actually have to --load-data for
>>>>> things
>>>>>> to
>>>>>>>>>> work. Again, these are integration tests.
>>>>>>>>>> - Too complex: Integration tests by their nature are
>> grabbing
>>>> too
>>>>>>>> much.
>>>>>>>>>> Mind you, I am not objecting to integration tests (I
>> actually
>>>> like
>>>>>>>> them)
>>>>>>>>>> but I am objecting to not having real unit-tests. Unit tests
>>>>> should
>>>>>>>> all
>>>>>>>>> run
>>>>>>>>>> in a few seconds.
>>>>>>>>>>
>>>>>>>>>> Proposed solution
>>>>>>>>>> --------------------------
>>>>>>>>>> - We keep what is considered real integration tests the way
>>> they
>>>>> are
>>>>>>>>> right
>>>>>>>>>> now and keep using them
>>>>>>>>>> - We move what should be unit tests into simple JUnit
>> classes,
>>>> and
>>>>>> we
>>>>>>>> do
>>>>>>>>>> not run them using java -jar ofbiz.jar --test, but instead
>> run
>>>>> them
>>>>>>>>>> directly from the build.xml script, so these files are not
>>>>>> identified
>>>>>>>> in
>>>>>>>>>> any XML document, but are simply called immediately from the
>>>> build
>>>>>>>>> scripts.
>>>>>>>>>> - We clearly mark the difference between integration tests
>> and
>>>>> unit
>>>>>>>> tests
>>>>>>>>>> (inside the source files or in the suite declarations).
>>>>>>>>>> - We change the run-tests target in build.xml to run both
>> unit
>>>>> tests
>>>>>>>> and
>>>>>>>>>> integration tests.
>>>>>>>>>>
>>>>>>>>>> I intend to heavily refactor the framework and I would feel
>>>> better
>>>>>>>> about
>>>>>>>>>> introducing this change while refactoring. What do you guys
>>>> think?
>>>>>>>> Ideas?
>>>>>>>>>> Suggestions? Approvals and thumbs up?
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>>
>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>
>>>>>>>

--
Ron Wheeler
President
Artifact Software Inc
email: [hidden email]
skype: ronaldmwheeler
phone: [phone], ext 102

[taher]

Hi Scott,

Thank you for your input. Your ideas are thought provoking and enriching to
the conversation.

The way I look at it, the general rule is usually many unit tests, less
integration tests, lesser functional tests. So we are not excluding any
types of test, all of them are important in certain areas for certain
purposes with certain quantity. Usually integration tests are less because
as you said they just grab more. I like the picture below as a general
guideline

http://i.stack.imgur.com/fjQvQ.png

As you already mentioned unit tests are useful for the framework. I
discovered errors in code that I wrote which I was very very careful with.
I immediately learned the lesson that humans are not designed to code, and
TDD gives you confidence as you build your code with those short 30-60
second red-green refactors. I feel much much safer and more confident
writing code that way, the test also documents how to use the api,
refactoring and feature change becomes less terrifying. Also messy code is
usually not test friendly thus refactoring and unit tests go hand-in-hand
for improving the code base.

Also, I am sorry to say that the framework code is rather messy and brittle
in many places. I think you probably encountered this yourself. The same is
said for the applications. Now If we start refactoring without unit tests
then we are back to scary business. So much can go wrong so fast and break
things you never expected to break. The framework with all applications
require heavy refactoring of things like massive ugly methods, sandwiched
logic, heavy shared mutable state, hidden dependencies, poor
interfaces, and much more. Every time I touch something I get shocked at
how bad it looks, spaghetti logic to no end. If you refactor with
integration tests instead of unit tests then you will come down to a
screeching halt as you wait between these test cycles. My computer actually
takes 10 minutes or more for a full clean load data and testing.

Now talking about mocking you raise some interesting points, what can you
mock vs utilizing integration tests? This is an excellent question that
does not have a simple answer, and you mostly learn as you code, difficult
to envision without coding. However simple things like Java services can be
mocked with a standard mocking class that you use everywhere. Transactions
are difficult to mock and better left to integration tests I think. SECAs
might be mocked by simply passing output to input as chaining methods. Mind
you I am not 100% sure of all of this, coding is the ultimate guide to what
can or cannot be done.

So I am not suggesting unit tests as a best-practice (even though it is).
Instead I suggest it as something that I and others did and got a huge
psychological relief and confidence and comfort from. Swimming in code
without tests is a terrifying business, made more terrifying if the code is
bad. Short bursts of red/green/red/green makes you feel good as you build
up your logic. And I don't have an exact vision of how to do it, because
the details always win.

I look forward to hearing your thoughts and thank you for enriching this
conversation.

Taher Alkhateeb

On Tuesday, 19 July 2016, Scott Gray <[hidden email]> wrote:

> I know I'm late to the party here, but I just want to say that I think
> integration tests have far greater value to OFBiz than unit tests.  Mostly
> because we tend to have quite a low number of tests and integration tests
> give us much better coverage per line of test code and the tests are much
> closer to the real world scenarios the application might encounter.
>
> I don't really see how unit tests could be applied to non-framework testing
> in a useful manner, could you expand on your vision in that regard?  I mean
> would we be testing something smaller than a service as the 'unit'?  What
> would we mock? Would transaction management still be active? What happens
> when the service calls another service, I guess we mock the response from
> that service (how)?
>
> It just seems a very complicated method to achieve a less thorough but
> albeit faster (maybe) test result.
>
> A build, data load and full test run takes 4m 9s on my laptop.  Excluding
> build, data load and framework tests: the application level tests take 35s,
> not very expensive IMO.  The data load time can be reduced to practically
> nothing by copying a clean slate database into runtime for each run.
>
> I'm mostly just suggesting we be wary of adding complicated testing
> procedures in the hope of achieving some 'best practice' result which in
> reality will provide minimal benefits.
>
> Regards
> Scott
>
> On 18 July 2016 at 18:57, Taher Alkhateeb <[hidden email]
> <javascript:;>>
> wrote:
>
> > Hello Akash,
> >
> > Fantastic, I have a few unit tests almost done to be included in the
> start
> > component. I will create a new subtask under OFBIZ-1463 to commit the
> tests
> > so you can use them as a reference if you like to.
> >
> > I also recommend that you follow the same directory structure between the
> > test code and production code. So for example:
> >
> > Production code:
> framework/start/src/main/java/org/apache/ofbiz/base/start
> > Test code: framework/start/src/test/java/org/apache/ofbiz/base/start
> >
> > The benefit of this hierarchy is that you can access non-public (package
> > protected) methods for testing. This is in fact exactly what I needed to
> be
> > able to apply some of the tests.
> >
> > Cheers,
> >
> > Taher Alkhateeb
> >
> > On Mon, Jul 18, 2016 at 9:22 AM, Akash Jain <
> [hidden email] <javascript:;>>
> > wrote:
> >
> > > Thanks Taher for nice initiative!
> > >
> > > We are planning to written unit tests to all components under
> OFBIZ-1463
> > >
> > > Thanks and Regards
> > > --
> > > Akash Jain
> > >
> > > On Mon, Jul 18, 2016 at 10:36 AM, Taher Alkhateeb <
> > > [hidden email] <javascript:;>> wrote:
> > >
> > > > Hello Everyone,
> > > >
> > > > In reference to this thread and the Jira OFBIZ-7254, I'm very happy
> to
> > > > announce that OFBiz is now ready for applying unit tests with only 8
> > new
> > > > lines of code in the build script (r1753143) :)
> > > >
> > > > I recommend we do the following moving forward:
> > > >
> > > > 1- Introduce unit tests as much as we can to all components
> > > > 2- Migrate most of the integration tests we currently have to unit
> > tests
> > > > (since they are designed to do very little integration).
> > > >
> > > > This is a great chance for us to practice real TDD in OFBiz
> > > >
> > > > Cheers,
> > > >
> > > > Taher Alkhateeb
> > > >
> > > > On Mon, Jun 13, 2016 at 7:43 AM, Pranay Pandey <
> > > > [hidden email] <javascript:;>> wrote:
> > > >
> > > > > +1
> > > > >
> > > > > Best regards,
> > > > >
> > > > > Pranay Pandey
> > > > > HotWax Systems
> > > > > http://www.hotwaxsystems.com/
> > > > >
> > > > > On Fri, Jun 10, 2016 at 7:46 PM, Taher Alkhateeb <
> > > > > [hidden email] <javascript:;>
> > > > > > wrote:
> > > > >
> > > > > > Hello Everyone,
> > > > > >
> > > > > > I was able to get a few tests running and this is very doable.
> But
> > I
> > > > > faced
> > > > > > a big problem in designing the testing framework because of ANT.
> > > > > >
> > > > > > The problem
> > > > > > ----------------
> > > > > > The way the build scripts are designed in OFBiz are very
> complex. A
> > > > > master
> > > > > > file calls other files which call other files. And in the middle
> > you
> > > > have
> > > > > > external libraries (ant-contrib) and macros, and variables, and
> > class
> > > > > path
> > > > > > declarations, and and and ....
> > > > > >
> > > > > > I cannot declare the tests programmatically (with JUnit test
> > suites)
> > > > > > because this means lower level components would depend on higher
> > > level
> > > > > > components. So I have to do it in ANT, by navigating this maze of
> > > build
> > > > > > scripts, and it was a headache for me just to read them, let
> alone
> > > > modify
> > > > > > them to create a testing framework.
> > > > > >
> > > > > > Suggested Solution
> > > > > > ------------------------
> > > > > > I suggest to implement the testing framework in Gradle, and
> simply
> > > call
> > > > > it
> > > > > > from within ant. This is a middle solution that sustains ant for
> > now,
> > > > but
> > > > > > can allow us to switch out later.
> > > > > >
> > > > > > This means I will just add one more file called build.gradle in
> the
> > > top
> > > > > > level directory, and figure out the business logic for calling
> the
> > > test
> > > > > > suites from that file
> > > > > >
> > > > > > I look forward to your feedback.
> > > > > >
> > > > > > Regards,
> > > > > >
> > > > > > Taher Alkhateeb
> > > > > >
> > > > > > On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <
> > > > > > [hidden email] <javascript:;>>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Everyone,
> > > > > > >
> > > > > > > Thank you all for your support, JIRA created in
> > > > > > > https://issues.apache.org/jira/browse/OFBIZ-7254
> > > > > > >
> > > > > > > I will start working on it and try to implement ASAP to get my
> > > focus
> > > > > back
> > > > > > > on refactoring.
> > > > > > >
> > > > > > > Cheers!
> > > > > > >
> > > > > > > Taher Alkhateeb
> > > > > > >
> > > > > > > On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
> > > > > > > [hidden email] <javascript:;>> wrote:
> > > > > > >
> > > > > > >> +1
> > > > > > >>
> > > > > > >> Thanks & Regards
> > > > > > >> --
> > > > > > >> Deepak Dixit
> > > > > > >> www.hotwaxsystems.com
> > > > > > >>
> > > > > > >> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
> > > > > > >> [hidden email] <javascript:;>> wrote:
> > > > > > >>
> > > > > > >> > +1
> > > > > > >> >
> > > > > > >> > Makes perfect sense.
> > > > > > >> >
> > > > > > >> > --
> > > > > > >> > Thanks & Regards,
> > > > > > >> > Mridul Pathak
> > > > > > >> > Senior Manager
> > > > > > >> > HotWax Systems
> > > > > > >> > http://www.hotwaxsystems.com
> > > > > > >> >
> > > > > > >> > > On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
> > > > > > >> [hidden email] <javascript:;>>
> > > > > > >> > wrote:
> > > > > > >> > >
> > > > > > >> > > Hello Everyone,
> > > > > > >> > >
> > > > > > >> > > After refactoring the start component and while starting
> on
> > > the
> > > > > base
> > > > > > >> > > component I realized that the testing framework for OFBiz
> is
> > > not
> > > > > > good.
> > > > > > >> > You
> > > > > > >> > > cannot do real test driven development or
> red-green-refactor
> > > > with
> > > > > > the
> > > > > > >> > > current setup, hence my proposal to change it. I explain
> > > below:
> > > > > > >> > >
> > > > > > >> > > Problem with current design
> > > > > > >> > > ----------------------------------------
> > > > > > >> > > - What we have right now is not unit tests, it's really
> > > > > integration
> > > > > > >> > tests.
> > > > > > >> > > You have to start the framework, the database, the service
> > > > engine,
> > > > > > the
> > > > > > >> > > entity engine and pretty much everything.
> > > > > > >> > > - Testing is very slow, because it's an integration test
> as
> > I
> > > > > > >> mentioned
> > > > > > >> > > above. 10 minutes on a good computer!
> > > > > > >> > > - There is zero mocking! We actually have to --load-data
> for
> > > > > things
> > > > > > to
> > > > > > >> > > work. Again, these are integration tests.
> > > > > > >> > > - Too complex: Integration tests by their nature are
> > grabbing
> > > > too
> > > > > > >> much.
> > > > > > >> > > Mind you, I am not objecting to integration tests (I
> > actually
> > > > like
> > > > > > >> them)
> > > > > > >> > > but I am objecting to not having real unit-tests. Unit
> tests
> > > > > should
> > > > > > >> all
> > > > > > >> > run
> > > > > > >> > > in a few seconds.
> > > > > > >> > >
> > > > > > >> > > Proposed solution
> > > > > > >> > > --------------------------
> > > > > > >> > > - We keep what is considered real integration tests the
> way
> > > they
> > > > > are
> > > > > > >> > right
> > > > > > >> > > now and keep using them
> > > > > > >> > > - We move what should be unit tests into simple JUnit
> > classes,
> > > > and
> > > > > > we
> > > > > > >> do
> > > > > > >> > > not run them using java -jar ofbiz.jar --test, but instead
> > run
> > > > > them
> > > > > > >> > > directly from the build.xml script, so these files are not
> > > > > > identified
> > > > > > >> in
> > > > > > >> > > any XML document, but are simply called immediately from
> the
> > > > build
> > > > > > >> > scripts.
> > > > > > >> > > - We clearly mark the difference between integration tests
> > and
> > > > > unit
> > > > > > >> tests
> > > > > > >> > > (inside the source files or in the suite declarations).
> > > > > > >> > > - We change the run-tests target in build.xml to run both
> > unit
> > > > > tests
> > > > > > >> and
> > > > > > >> > > integration tests.
> > > > > > >> > >
> > > > > > >> > > I intend to heavily refactor the framework and I would
> feel
> > > > better
> > > > > > >> about
> > > > > > >> > > introducing this change while refactoring. What do you
> guys
> > > > think?
> > > > > > >> Ideas?
> > > > > > >> > > Suggestions? Approvals and thumbs up?
> > > > > > >> > >
> > > > > > >> > > Regards,
> > > > > > >> > >
> > > > > > >> > > Taher Alkhateeb
> > > > > > >> >
> > > > > > >> >
> > > > > > >>
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

[Scott Gray-3]

Thanks Ron and Taher for your responses, I appreciate them but I don't hear
much in the conversation that speaks to the value of unit tests over
integration tests.  Most of the thoughts shared speaks more to the value of
tests rather than differences in the type of tests.

Speed: At an application level we have ~685 tests that run in 35 seconds
(excluding build and data load).  Another point is that there isn't much
reason why tests can't be run in parallel rather than sequentially.
TDD: Integration tests in OFBiz are as simple if not simpler to write than
unit tests and just as useful for TDD.
I'm not sure if I missed any other points raised regarding integration vs.
unit tests?

I'm not looking to start a big long debate on the topic, I just wanted to
speak out that someone out there (me) doesn't think unit tests are the best
solution for testing OFBiz applications.

Regards
Scott

On 19 July 2016 at 16:52, Taher Alkhateeb <[hidden email]>
wrote:

> Hi Scott,
>
> Thank you for your input. Your ideas are thought provoking and enriching to
> the conversation.
>
> The way I look at it, the general rule is usually many unit tests, less
> integration tests, lesser functional tests. So we are not excluding any
> types of test, all of them are important in certain areas for certain
> purposes with certain quantity. Usually integration tests are less because
> as you said they just grab more. I like the picture below as a general
> guideline
>
> http://i.stack.imgur.com/fjQvQ.png
>
> As you already mentioned unit tests are useful for the framework. I
> discovered errors in code that I wrote which I was very very careful with.
> I immediately learned the lesson that humans are not designed to code, and
> TDD gives you confidence as you build your code with those short 30-60
> second red-green refactors. I feel much much safer and more confident
> writing code that way, the test also documents how to use the api,
> refactoring and feature change becomes less terrifying. Also messy code is
> usually not test friendly thus refactoring and unit tests go hand-in-hand
> for improving the code base.
>
> Also, I am sorry to say that the framework code is rather messy and brittle
> in many places. I think you probably encountered this yourself. The same is
> said for the applications. Now If we start refactoring without unit tests
> then we are back to scary business. So much can go wrong so fast and break
> things you never expected to break. The framework with all applications
> require heavy refactoring of things like massive ugly methods, sandwiched
> logic, heavy shared mutable state, hidden dependencies, poor
> interfaces, and much more. Every time I touch something I get shocked at
> how bad it looks, spaghetti logic to no end. If you refactor with
> integration tests instead of unit tests then you will come down to a
> screeching halt as you wait between these test cycles. My computer actually
> takes 10 minutes or more for a full clean load data and testing.
>
> Now talking about mocking you raise some interesting points, what can you
> mock vs utilizing integration tests? This is an excellent question that
> does not have a simple answer, and you mostly learn as you code, difficult
> to envision without coding. However simple things like Java services can be
> mocked with a standard mocking class that you use everywhere. Transactions
> are difficult to mock and better left to integration tests I think. SECAs
> might be mocked by simply passing output to input as chaining methods. Mind
> you I am not 100% sure of all of this, coding is the ultimate guide to what
> can or cannot be done.
>
> So I am not suggesting unit tests as a best-practice (even though it is).
> Instead I suggest it as something that I and others did and got a huge
> psychological relief and confidence and comfort from. Swimming in code
> without tests is a terrifying business, made more terrifying if the code is
> bad. Short bursts of red/green/red/green makes you feel good as you build
> up your logic. And I don't have an exact vision of how to do it, because
> the details always win.
>
> I look forward to hearing your thoughts and thank you for enriching this
> conversation.
>
> Taher Alkhateeb
>
> On Tuesday, 19 July 2016, Scott Gray <[hidden email]> wrote:
>
> > I know I'm late to the party here, but I just want to say that I think
> > integration tests have far greater value to OFBiz than unit tests.
> Mostly
> > because we tend to have quite a low number of tests and integration tests
> > give us much better coverage per line of test code and the tests are much
> > closer to the real world scenarios the application might encounter.
> >
> > I don't really see how unit tests could be applied to non-framework
> testing
> > in a useful manner, could you expand on your vision in that regard?  I
> mean
> > would we be testing something smaller than a service as the 'unit'?  What
> > would we mock? Would transaction management still be active? What happens
> > when the service calls another service, I guess we mock the response from
> > that service (how)?
> >
> > It just seems a very complicated method to achieve a less thorough but
> > albeit faster (maybe) test result.
> >
> > A build, data load and full test run takes 4m 9s on my laptop.  Excluding
> > build, data load and framework tests: the application level tests take
> 35s,
> > not very expensive IMO.  The data load time can be reduced to practically
> > nothing by copying a clean slate database into runtime for each run.
> >
> > I'm mostly just suggesting we be wary of adding complicated testing
> > procedures in the hope of achieving some 'best practice' result which in
> > reality will provide minimal benefits.
> >
> > Regards
> > Scott
> >
> > On 18 July 2016 at 18:57, Taher Alkhateeb <[hidden email]
> > <javascript:;>>
> > wrote:
> >
> > > Hello Akash,
> > >
> > > Fantastic, I have a few unit tests almost done to be included in the
> > start
> > > component. I will create a new subtask under OFBIZ-1463 to commit the
> > tests
> > > so you can use them as a reference if you like to.
> > >
> > > I also recommend that you follow the same directory structure between
> the
> > > test code and production code. So for example:
> > >
> > > Production code:
> > framework/start/src/main/java/org/apache/ofbiz/base/start
> > > Test code: framework/start/src/test/java/org/apache/ofbiz/base/start
> > >
> > > The benefit of this hierarchy is that you can access non-public
> (package
> > > protected) methods for testing. This is in fact exactly what I needed
> to
> > be
> > > able to apply some of the tests.
> > >
> > > Cheers,
> > >
> > > Taher Alkhateeb
> > >
> > > On Mon, Jul 18, 2016 at 9:22 AM, Akash Jain <
> > [hidden email] <javascript:;>>
> > > wrote:
> > >
> > > > Thanks Taher for nice initiative!
> > > >
> > > > We are planning to written unit tests to all components under
> > OFBIZ-1463
> > > >
> > > > Thanks and Regards
> > > > --
> > > > Akash Jain
> > > >
> > > > On Mon, Jul 18, 2016 at 10:36 AM, Taher Alkhateeb <
> > > > [hidden email] <javascript:;>> wrote:
> > > >
> > > > > Hello Everyone,
> > > > >
> > > > > In reference to this thread and the Jira OFBIZ-7254, I'm very happy
> > to
> > > > > announce that OFBiz is now ready for applying unit tests with only
> 8
> > > new
> > > > > lines of code in the build script (r1753143) :)
> > > > >
> > > > > I recommend we do the following moving forward:
> > > > >
> > > > > 1- Introduce unit tests as much as we can to all components
> > > > > 2- Migrate most of the integration tests we currently have to unit
> > > tests
> > > > > (since they are designed to do very little integration).
> > > > >
> > > > > This is a great chance for us to practice real TDD in OFBiz
> > > > >
> > > > > Cheers,
> > > > >
> > > > > Taher Alkhateeb
> > > > >
> > > > > On Mon, Jun 13, 2016 at 7:43 AM, Pranay Pandey <
> > > > > [hidden email] <javascript:;>> wrote:
> > > > >
> > > > > > +1
> > > > > >
> > > > > > Best regards,
> > > > > >
> > > > > > Pranay Pandey
> > > > > > HotWax Systems
> > > > > > http://www.hotwaxsystems.com/
> > > > > >
> > > > > > On Fri, Jun 10, 2016 at 7:46 PM, Taher Alkhateeb <
> > > > > > [hidden email] <javascript:;>
> > > > > > > wrote:
> > > > > >
> > > > > > > Hello Everyone,
> > > > > > >
> > > > > > > I was able to get a few tests running and this is very doable.
> > But
> > > I
> > > > > > faced
> > > > > > > a big problem in designing the testing framework because of
> ANT.
> > > > > > >
> > > > > > > The problem
> > > > > > > ----------------
> > > > > > > The way the build scripts are designed in OFBiz are very
> > complex. A
> > > > > > master
> > > > > > > file calls other files which call other files. And in the
> middle
> > > you
> > > > > have
> > > > > > > external libraries (ant-contrib) and macros, and variables, and
> > > class
> > > > > > path
> > > > > > > declarations, and and and ....
> > > > > > >
> > > > > > > I cannot declare the tests programmatically (with JUnit test
> > > suites)
> > > > > > > because this means lower level components would depend on
> higher
> > > > level
> > > > > > > components. So I have to do it in ANT, by navigating this maze
> of
> > > > build
> > > > > > > scripts, and it was a headache for me just to read them, let
> > alone
> > > > > modify
> > > > > > > them to create a testing framework.
> > > > > > >
> > > > > > > Suggested Solution
> > > > > > > ------------------------
> > > > > > > I suggest to implement the testing framework in Gradle, and
> > simply
> > > > call
> > > > > > it
> > > > > > > from within ant. This is a middle solution that sustains ant
> for
> > > now,
> > > > > but
> > > > > > > can allow us to switch out later.
> > > > > > >
> > > > > > > This means I will just add one more file called build.gradle in
> > the
> > > > top
> > > > > > > level directory, and figure out the business logic for calling
> > the
> > > > test
> > > > > > > suites from that file
> > > > > > >
> > > > > > > I look forward to your feedback.
> > > > > > >
> > > > > > > Regards,
> > > > > > >
> > > > > > > Taher Alkhateeb
> > > > > > >
> > > > > > > On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <
> > > > > > > [hidden email] <javascript:;>>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi Everyone,
> > > > > > > >
> > > > > > > > Thank you all for your support, JIRA created in
> > > > > > > > https://issues.apache.org/jira/browse/OFBIZ-7254
> > > > > > > >
> > > > > > > > I will start working on it and try to implement ASAP to get
> my
> > > > focus
> > > > > > back
> > > > > > > > on refactoring.
> > > > > > > >
> > > > > > > > Cheers!
> > > > > > > >
> > > > > > > > Taher Alkhateeb
> > > > > > > >
> > > > > > > > On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
> > > > > > > > [hidden email] <javascript:;>> wrote:
> > > > > > > >
> > > > > > > >> +1
> > > > > > > >>
> > > > > > > >> Thanks & Regards
> > > > > > > >> --
> > > > > > > >> Deepak Dixit
> > > > > > > >> www.hotwaxsystems.com
> > > > > > > >>
> > > > > > > >> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
> > > > > > > >> [hidden email] <javascript:;>> wrote:
> > > > > > > >>
> > > > > > > >> > +1
> > > > > > > >> >
> > > > > > > >> > Makes perfect sense.
> > > > > > > >> >
> > > > > > > >> > --
> > > > > > > >> > Thanks & Regards,
> > > > > > > >> > Mridul Pathak
> > > > > > > >> > Senior Manager
> > > > > > > >> > HotWax Systems
> > > > > > > >> > http://www.hotwaxsystems.com
> > > > > > > >> >
> > > > > > > >> > > On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
> > > > > > > >> [hidden email] <javascript:;>>
> > > > > > > >> > wrote:
> > > > > > > >> > >
> > > > > > > >> > > Hello Everyone,
> > > > > > > >> > >
> > > > > > > >> > > After refactoring the start component and while starting
> > on
> > > > the
> > > > > > base
> > > > > > > >> > > component I realized that the testing framework for
> OFBiz
> > is
> > > > not
> > > > > > > good.
> > > > > > > >> > You
> > > > > > > >> > > cannot do real test driven development or
> > red-green-refactor
> > > > > with
> > > > > > > the
> > > > > > > >> > > current setup, hence my proposal to change it. I explain
> > > > below:
> > > > > > > >> > >
> > > > > > > >> > > Problem with current design
> > > > > > > >> > > ----------------------------------------
> > > > > > > >> > > - What we have right now is not unit tests, it's really
> > > > > > integration
> > > > > > > >> > tests.
> > > > > > > >> > > You have to start the framework, the database, the
> service
> > > > > engine,
> > > > > > > the
> > > > > > > >> > > entity engine and pretty much everything.
> > > > > > > >> > > - Testing is very slow, because it's an integration test
> > as
> > > I
> > > > > > > >> mentioned
> > > > > > > >> > > above. 10 minutes on a good computer!
> > > > > > > >> > > - There is zero mocking! We actually have to --load-data
> > for
> > > > > > things
> > > > > > > to
> > > > > > > >> > > work. Again, these are integration tests.
> > > > > > > >> > > - Too complex: Integration tests by their nature are
> > > grabbing
> > > > > too
> > > > > > > >> much.
> > > > > > > >> > > Mind you, I am not objecting to integration tests (I
> > > actually
> > > > > like
> > > > > > > >> them)
> > > > > > > >> > > but I am objecting to not having real unit-tests. Unit
> > tests
> > > > > > should
> > > > > > > >> all
> > > > > > > >> > run
> > > > > > > >> > > in a few seconds.
> > > > > > > >> > >
> > > > > > > >> > > Proposed solution
> > > > > > > >> > > --------------------------
> > > > > > > >> > > - We keep what is considered real integration tests the
> > way
> > > > they
> > > > > > are
> > > > > > > >> > right
> > > > > > > >> > > now and keep using them
> > > > > > > >> > > - We move what should be unit tests into simple JUnit
> > > classes,
> > > > > and
> > > > > > > we
> > > > > > > >> do
> > > > > > > >> > > not run them using java -jar ofbiz.jar --test, but
> instead
> > > run
> > > > > > them
> > > > > > > >> > > directly from the build.xml script, so these files are
> not
> > > > > > > identified
> > > > > > > >> in
> > > > > > > >> > > any XML document, but are simply called immediately from
> > the
> > > > > build
> > > > > > > >> > scripts.
> > > > > > > >> > > - We clearly mark the difference between integration
> tests
> > > and
> > > > > > unit
> > > > > > > >> tests
> > > > > > > >> > > (inside the source files or in the suite declarations).
> > > > > > > >> > > - We change the run-tests target in build.xml to run
> both
> > > unit
> > > > > > tests
> > > > > > > >> and
> > > > > > > >> > > integration tests.
> > > > > > > >> > >
> > > > > > > >> > > I intend to heavily refactor the framework and I would
> > feel
> > > > > better
> > > > > > > >> about
> > > > > > > >> > > introducing this change while refactoring. What do you
> > guys
> > > > > think?
> > > > > > > >> Ideas?
> > > > > > > >> > > Suggestions? Approvals and thumbs up?
> > > > > > > >> > >
> > > > > > > >> > > Regards,
> > > > > > > >> > >
> > > > > > > >> > > Taher Alkhateeb
> > > > > > > >> >
> > > > > > > >> >
> > > > > > > >>
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

[taher]

Hi Scott,

Thank you for the feedback. To be focused exactly on integration vs unit, I
already mentioned above that at least for me the main objective is to
confidently and quickly run the tests in short bursts of red-green
refactor. This allows me to refactor code without waiting in front of the
screen in between test cycles thus giving me immediate feedback on any
errors I made. Perhaps my intro was too long so this is the squeezed out
version of it.

I already had one round of testing in the start component which was much
faster that way and had an immediate impact. Oh and by the way, you cannot
test the start component with integration tests for example unless you do
it from an external component which cannot access package protected items.

This style of coding is applicable I think to any software project
inclusive of OFBiz. Maybe in certain components more than others or certain
areas more than others. But I can't see how it could be bad or unwanted.

Taher Alkhateeb

On Jul 19, 2016 8:18 AM, "Scott Gray" <[hidden email]> wrote:

> Thanks Ron and Taher for your responses, I appreciate them but I don't hear
> much in the conversation that speaks to the value of unit tests over
> integration tests.  Most of the thoughts shared speaks more to the value of
> tests rather than differences in the type of tests.
>
> Speed: At an application level we have ~685 tests that run in 35 seconds
> (excluding build and data load).  Another point is that there isn't much
> reason why tests can't be run in parallel rather than sequentially.
> TDD: Integration tests in OFBiz are as simple if not simpler to write than
> unit tests and just as useful for TDD.
> I'm not sure if I missed any other points raised regarding integration vs.
> unit tests?
>
> I'm not looking to start a big long debate on the topic, I just wanted to
> speak out that someone out there (me) doesn't think unit tests are the best
> solution for testing OFBiz applications.
>
> Regards
> Scott
>
> On 19 July 2016 at 16:52, Taher Alkhateeb <[hidden email]>
> wrote:
>
> > Hi Scott,
> >
> > Thank you for your input. Your ideas are thought provoking and enriching
> to
> > the conversation.
> >
> > The way I look at it, the general rule is usually many unit tests, less
> > integration tests, lesser functional tests. So we are not excluding any
> > types of test, all of them are important in certain areas for certain
> > purposes with certain quantity. Usually integration tests are less
> because
> > as you said they just grab more. I like the picture below as a general
> > guideline
> >
> > http://i.stack.imgur.com/fjQvQ.png
> >
> > As you already mentioned unit tests are useful for the framework. I
> > discovered errors in code that I wrote which I was very very careful
> with.
> > I immediately learned the lesson that humans are not designed to code,
> and
> > TDD gives you confidence as you build your code with those short 30-60
> > second red-green refactors. I feel much much safer and more confident
> > writing code that way, the test also documents how to use the api,
> > refactoring and feature change becomes less terrifying. Also messy code
> is
> > usually not test friendly thus refactoring and unit tests go hand-in-hand
> > for improving the code base.
> >
> > Also, I am sorry to say that the framework code is rather messy and
> brittle
> > in many places. I think you probably encountered this yourself. The same
> is
> > said for the applications. Now If we start refactoring without unit tests
> > then we are back to scary business. So much can go wrong so fast and
> break
> > things you never expected to break. The framework with all applications
> > require heavy refactoring of things like massive ugly methods, sandwiched
> > logic, heavy shared mutable state, hidden dependencies, poor
> > interfaces, and much more. Every time I touch something I get shocked at
> > how bad it looks, spaghetti logic to no end. If you refactor with
> > integration tests instead of unit tests then you will come down to a
> > screeching halt as you wait between these test cycles. My computer
> actually
> > takes 10 minutes or more for a full clean load data and testing.
> >
> > Now talking about mocking you raise some interesting points, what can you
> > mock vs utilizing integration tests? This is an excellent question that
> > does not have a simple answer, and you mostly learn as you code,
> difficult
> > to envision without coding. However simple things like Java services can
> be
> > mocked with a standard mocking class that you use everywhere.
> Transactions
> > are difficult to mock and better left to integration tests I think. SECAs
> > might be mocked by simply passing output to input as chaining methods.
> Mind
> > you I am not 100% sure of all of this, coding is the ultimate guide to
> what
> > can or cannot be done.
> >
> > So I am not suggesting unit tests as a best-practice (even though it is).
> > Instead I suggest it as something that I and others did and got a huge
> > psychological relief and confidence and comfort from. Swimming in code
> > without tests is a terrifying business, made more terrifying if the code
> is
> > bad. Short bursts of red/green/red/green makes you feel good as you build
> > up your logic. And I don't have an exact vision of how to do it, because
> > the details always win.
> >
> > I look forward to hearing your thoughts and thank you for enriching this
> > conversation.
> >
> > Taher Alkhateeb
> >
> > On Tuesday, 19 July 2016, Scott Gray <[hidden email]>
> wrote:
> >
> > > I know I'm late to the party here, but I just want to say that I think
> > > integration tests have far greater value to OFBiz than unit tests.
> > Mostly
> > > because we tend to have quite a low number of tests and integration
> tests
> > > give us much better coverage per line of test code and the tests are
> much
> > > closer to the real world scenarios the application might encounter.
> > >
> > > I don't really see how unit tests could be applied to non-framework
> > testing
> > > in a useful manner, could you expand on your vision in that regard?  I
> > mean
> > > would we be testing something smaller than a service as the 'unit'?
> What
> > > would we mock? Would transaction management still be active? What
> happens
> > > when the service calls another service, I guess we mock the response
> from
> > > that service (how)?
> > >
> > > It just seems a very complicated method to achieve a less thorough but
> > > albeit faster (maybe) test result.
> > >
> > > A build, data load and full test run takes 4m 9s on my laptop.
> Excluding
> > > build, data load and framework tests: the application level tests take
> > 35s,
> > > not very expensive IMO.  The data load time can be reduced to
> practically
> > > nothing by copying a clean slate database into runtime for each run.
> > >
> > > I'm mostly just suggesting we be wary of adding complicated testing
> > > procedures in the hope of achieving some 'best practice' result which
> in
> > > reality will provide minimal benefits.
> > >
> > > Regards
> > > Scott
> > >
> > > On 18 July 2016 at 18:57, Taher Alkhateeb <[hidden email]
> > > <javascript:;>>
> > > wrote:
> > >
> > > > Hello Akash,
> > > >
> > > > Fantastic, I have a few unit tests almost done to be included in the
> > > start
> > > > component. I will create a new subtask under OFBIZ-1463 to commit the
> > > tests
> > > > so you can use them as a reference if you like to.
> > > >
> > > > I also recommend that you follow the same directory structure between
> > the
> > > > test code and production code. So for example:
> > > >
> > > > Production code:
> > > framework/start/src/main/java/org/apache/ofbiz/base/start
> > > > Test code: framework/start/src/test/java/org/apache/ofbiz/base/start
> > > >
> > > > The benefit of this hierarchy is that you can access non-public
> > (package
> > > > protected) methods for testing. This is in fact exactly what I needed
> > to
> > > be
> > > > able to apply some of the tests.
> > > >
> > > > Cheers,
> > > >
> > > > Taher Alkhateeb
> > > >
> > > > On Mon, Jul 18, 2016 at 9:22 AM, Akash Jain <
> > > [hidden email] <javascript:;>>
> > > > wrote:
> > > >
> > > > > Thanks Taher for nice initiative!
> > > > >
> > > > > We are planning to written unit tests to all components under
> > > OFBIZ-1463
> > > > >
> > > > > Thanks and Regards
> > > > > --
> > > > > Akash Jain
> > > > >
> > > > > On Mon, Jul 18, 2016 at 10:36 AM, Taher Alkhateeb <
> > > > > [hidden email] <javascript:;>> wrote:
> > > > >
> > > > > > Hello Everyone,
> > > > > >
> > > > > > In reference to this thread and the Jira OFBIZ-7254, I'm very
> happy
> > > to
> > > > > > announce that OFBiz is now ready for applying unit tests with
> only
> > 8
> > > > new
> > > > > > lines of code in the build script (r1753143) :)
> > > > > >
> > > > > > I recommend we do the following moving forward:
> > > > > >
> > > > > > 1- Introduce unit tests as much as we can to all components
> > > > > > 2- Migrate most of the integration tests we currently have to
> unit
> > > > tests
> > > > > > (since they are designed to do very little integration).
> > > > > >
> > > > > > This is a great chance for us to practice real TDD in OFBiz
> > > > > >
> > > > > > Cheers,
> > > > > >
> > > > > > Taher Alkhateeb
> > > > > >
> > > > > > On Mon, Jun 13, 2016 at 7:43 AM, Pranay Pandey <
> > > > > > [hidden email] <javascript:;>> wrote:
> > > > > >
> > > > > > > +1
> > > > > > >
> > > > > > > Best regards,
> > > > > > >
> > > > > > > Pranay Pandey
> > > > > > > HotWax Systems
> > > > > > > http://www.hotwaxsystems.com/
> > > > > > >
> > > > > > > On Fri, Jun 10, 2016 at 7:46 PM, Taher Alkhateeb <
> > > > > > > [hidden email] <javascript:;>
> > > > > > > > wrote:
> > > > > > >
> > > > > > > > Hello Everyone,
> > > > > > > >
> > > > > > > > I was able to get a few tests running and this is very
> doable.
> > > But
> > > > I
> > > > > > > faced
> > > > > > > > a big problem in designing the testing framework because of
> > ANT.
> > > > > > > >
> > > > > > > > The problem
> > > > > > > > ----------------
> > > > > > > > The way the build scripts are designed in OFBiz are very
> > > complex. A
> > > > > > > master
> > > > > > > > file calls other files which call other files. And in the
> > middle
> > > > you
> > > > > > have
> > > > > > > > external libraries (ant-contrib) and macros, and variables,
> and
> > > > class
> > > > > > > path
> > > > > > > > declarations, and and and ....
> > > > > > > >
> > > > > > > > I cannot declare the tests programmatically (with JUnit test
> > > > suites)
> > > > > > > > because this means lower level components would depend on
> > higher
> > > > > level
> > > > > > > > components. So I have to do it in ANT, by navigating this
> maze
> > of
> > > > > build
> > > > > > > > scripts, and it was a headache for me just to read them, let
> > > alone
> > > > > > modify
> > > > > > > > them to create a testing framework.
> > > > > > > >
> > > > > > > > Suggested Solution
> > > > > > > > ------------------------
> > > > > > > > I suggest to implement the testing framework in Gradle, and
> > > simply
> > > > > call
> > > > > > > it
> > > > > > > > from within ant. This is a middle solution that sustains ant
> > for
> > > > now,
> > > > > > but
> > > > > > > > can allow us to switch out later.
> > > > > > > >
> > > > > > > > This means I will just add one more file called build.gradle
> in
> > > the
> > > > > top
> > > > > > > > level directory, and figure out the business logic for
> calling
> > > the
> > > > > test
> > > > > > > > suites from that file
> > > > > > > >
> > > > > > > > I look forward to your feedback.
> > > > > > > >
> > > > > > > > Regards,
> > > > > > > >
> > > > > > > > Taher Alkhateeb
> > > > > > > >
> > > > > > > > On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <
> > > > > > > > [hidden email] <javascript:;>>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi Everyone,
> > > > > > > > >
> > > > > > > > > Thank you all for your support, JIRA created in
> > > > > > > > > https://issues.apache.org/jira/browse/OFBIZ-7254
> > > > > > > > >
> > > > > > > > > I will start working on it and try to implement ASAP to get
> > my
> > > > > focus
> > > > > > > back
> > > > > > > > > on refactoring.
> > > > > > > > >
> > > > > > > > > Cheers!
> > > > > > > > >
> > > > > > > > > Taher Alkhateeb
> > > > > > > > >
> > > > > > > > > On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
> > > > > > > > > [hidden email] <javascript:;>> wrote:
> > > > > > > > >
> > > > > > > > >> +1
> > > > > > > > >>
> > > > > > > > >> Thanks & Regards
> > > > > > > > >> --
> > > > > > > > >> Deepak Dixit
> > > > > > > > >> www.hotwaxsystems.com
> > > > > > > > >>
> > > > > > > > >> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
> > > > > > > > >> [hidden email] <javascript:;>> wrote:
> > > > > > > > >>
> > > > > > > > >> > +1
> > > > > > > > >> >
> > > > > > > > >> > Makes perfect sense.
> > > > > > > > >> >
> > > > > > > > >> > --
> > > > > > > > >> > Thanks & Regards,
> > > > > > > > >> > Mridul Pathak
> > > > > > > > >> > Senior Manager
> > > > > > > > >> > HotWax Systems
> > > > > > > > >> > http://www.hotwaxsystems.com
> > > > > > > > >> >
> > > > > > > > >> > > On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
> > > > > > > > >> [hidden email] <javascript:;>>
> > > > > > > > >> > wrote:
> > > > > > > > >> > >
> > > > > > > > >> > > Hello Everyone,
> > > > > > > > >> > >
> > > > > > > > >> > > After refactoring the start component and while
> starting
> > > on
> > > > > the
> > > > > > > base
> > > > > > > > >> > > component I realized that the testing framework for
> > OFBiz
> > > is
> > > > > not
> > > > > > > > good.
> > > > > > > > >> > You
> > > > > > > > >> > > cannot do real test driven development or
> > > red-green-refactor
> > > > > > with
> > > > > > > > the
> > > > > > > > >> > > current setup, hence my proposal to change it. I
> explain
> > > > > below:
> > > > > > > > >> > >
> > > > > > > > >> > > Problem with current design
> > > > > > > > >> > > ----------------------------------------
> > > > > > > > >> > > - What we have right now is not unit tests, it's
> really
> > > > > > > integration
> > > > > > > > >> > tests.
> > > > > > > > >> > > You have to start the framework, the database, the
> > service
> > > > > > engine,
> > > > > > > > the
> > > > > > > > >> > > entity engine and pretty much everything.
> > > > > > > > >> > > - Testing is very slow, because it's an integration
> test
> > > as
> > > > I
> > > > > > > > >> mentioned
> > > > > > > > >> > > above. 10 minutes on a good computer!
> > > > > > > > >> > > - There is zero mocking! We actually have to
> --load-data
> > > for
> > > > > > > things
> > > > > > > > to
> > > > > > > > >> > > work. Again, these are integration tests.
> > > > > > > > >> > > - Too complex: Integration tests by their nature are
> > > > grabbing
> > > > > > too
> > > > > > > > >> much.
> > > > > > > > >> > > Mind you, I am not objecting to integration tests (I
> > > > actually
> > > > > > like
> > > > > > > > >> them)
> > > > > > > > >> > > but I am objecting to not having real unit-tests. Unit
> > > tests
> > > > > > > should
> > > > > > > > >> all
> > > > > > > > >> > run
> > > > > > > > >> > > in a few seconds.
> > > > > > > > >> > >
> > > > > > > > >> > > Proposed solution
> > > > > > > > >> > > --------------------------
> > > > > > > > >> > > - We keep what is considered real integration tests
> the
> > > way
> > > > > they
> > > > > > > are
> > > > > > > > >> > right
> > > > > > > > >> > > now and keep using them
> > > > > > > > >> > > - We move what should be unit tests into simple JUnit
> > > > classes,
> > > > > > and
> > > > > > > > we
> > > > > > > > >> do
> > > > > > > > >> > > not run them using java -jar ofbiz.jar --test, but
> > instead
> > > > run
> > > > > > > them
> > > > > > > > >> > > directly from the build.xml script, so these files are
> > not
> > > > > > > > identified
> > > > > > > > >> in
> > > > > > > > >> > > any XML document, but are simply called immediately
> from
> > > the
> > > > > > build
> > > > > > > > >> > scripts.
> > > > > > > > >> > > - We clearly mark the difference between integration
> > tests
> > > > and
> > > > > > > unit
> > > > > > > > >> tests
> > > > > > > > >> > > (inside the source files or in the suite
> declarations).
> > > > > > > > >> > > - We change the run-tests target in build.xml to run
> > both
> > > > unit
> > > > > > > tests
> > > > > > > > >> and
> > > > > > > > >> > > integration tests.
> > > > > > > > >> > >
> > > > > > > > >> > > I intend to heavily refactor the framework and I would
> > > feel
> > > > > > better
> > > > > > > > >> about
> > > > > > > > >> > > introducing this change while refactoring. What do you
> > > guys
> > > > > > think?
> > > > > > > > >> Ideas?
> > > > > > > > >> > > Suggestions? Approvals and thumbs up?
> > > > > > > > >> > >
> > > > > > > > >> > > Regards,
> > > > > > > > >> > >
> > > > > > > > >> > > Taher Alkhateeb
> > > > > > > > >> >
> > > > > > > > >> >
> > > > > > > > >>
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

[Scott Gray-3]

Yeah I'm sure unit tests probably worked well for the start component given
it is the lowest level component in the system and closest to a basic java
app.  I just think the value proposition might decrease the further up the
stack we move with them.  I'm not against unit tests when they prove
useful, but further up the stack I think we should prove the case for them
before doing much work to support the mocking that will be required to keep
them inline with 'best practices'.

In OFBiz "bad or unwanted" tends to come mostly in the way of increasing
complexity from features that require more effort to maintain than the
value they provide to the community.  I think there's a chance attempting
to mock service tests could fall into that bucket.  I could be wrong, but I
think it's worth looking into before we declare that unit tests are the
best form of testing for OFBiz.

Regards
Scott

On 19 July 2016 at 17:37, Taher Alkhateeb <[hidden email]>
wrote:

> Hi Scott,
>
> Thank you for the feedback. To be focused exactly on integration vs unit, I
> already mentioned above that at least for me the main objective is to
> confidently and quickly run the tests in short bursts of red-green
> refactor. This allows me to refactor code without waiting in front of the
> screen in between test cycles thus giving me immediate feedback on any
> errors I made. Perhaps my intro was too long so this is the squeezed out
> version of it.
>
> I already had one round of testing in the start component which was much
> faster that way and had an immediate impact. Oh and by the way, you cannot
> test the start component with integration tests for example unless you do
> it from an external component which cannot access package protected items.
>
> This style of coding is applicable I think to any software project
> inclusive of OFBiz. Maybe in certain components more than others or certain
> areas more than others. But I can't see how it could be bad or unwanted.
>
> Taher Alkhateeb
>
> On Jul 19, 2016 8:18 AM, "Scott Gray" <[hidden email]>
> wrote:
>
> > Thanks Ron and Taher for your responses, I appreciate them but I don't
> hear
> > much in the conversation that speaks to the value of unit tests over
> > integration tests.  Most of the thoughts shared speaks more to the value
> of
> > tests rather than differences in the type of tests.
> >
> > Speed: At an application level we have ~685 tests that run in 35 seconds
> > (excluding build and data load).  Another point is that there isn't much
> > reason why tests can't be run in parallel rather than sequentially.
> > TDD: Integration tests in OFBiz are as simple if not simpler to write
> than
> > unit tests and just as useful for TDD.
> > I'm not sure if I missed any other points raised regarding integration
> vs.
> > unit tests?
> >
> > I'm not looking to start a big long debate on the topic, I just wanted to
> > speak out that someone out there (me) doesn't think unit tests are the
> best
> > solution for testing OFBiz applications.
> >
> > Regards
> > Scott
> >
> > On 19 July 2016 at 16:52, Taher Alkhateeb <[hidden email]>
> > wrote:
> >
> > > Hi Scott,
> > >
> > > Thank you for your input. Your ideas are thought provoking and
> enriching
> > to
> > > the conversation.
> > >
> > > The way I look at it, the general rule is usually many unit tests, less
> > > integration tests, lesser functional tests. So we are not excluding any
> > > types of test, all of them are important in certain areas for certain
> > > purposes with certain quantity. Usually integration tests are less
> > because
> > > as you said they just grab more. I like the picture below as a general
> > > guideline
> > >
> > > http://i.stack.imgur.com/fjQvQ.png
> > >
> > > As you already mentioned unit tests are useful for the framework. I
> > > discovered errors in code that I wrote which I was very very careful
> > with.
> > > I immediately learned the lesson that humans are not designed to code,
> > and
> > > TDD gives you confidence as you build your code with those short 30-60
> > > second red-green refactors. I feel much much safer and more confident
> > > writing code that way, the test also documents how to use the api,
> > > refactoring and feature change becomes less terrifying. Also messy code
> > is
> > > usually not test friendly thus refactoring and unit tests go
> hand-in-hand
> > > for improving the code base.
> > >
> > > Also, I am sorry to say that the framework code is rather messy and
> > brittle
> > > in many places. I think you probably encountered this yourself. The
> same
> > is
> > > said for the applications. Now If we start refactoring without unit
> tests
> > > then we are back to scary business. So much can go wrong so fast and
> > break
> > > things you never expected to break. The framework with all applications
> > > require heavy refactoring of things like massive ugly methods,
> sandwiched
> > > logic, heavy shared mutable state, hidden dependencies, poor
> > > interfaces, and much more. Every time I touch something I get shocked
> at
> > > how bad it looks, spaghetti logic to no end. If you refactor with
> > > integration tests instead of unit tests then you will come down to a
> > > screeching halt as you wait between these test cycles. My computer
> > actually
> > > takes 10 minutes or more for a full clean load data and testing.
> > >
> > > Now talking about mocking you raise some interesting points, what can
> you
> > > mock vs utilizing integration tests? This is an excellent question that
> > > does not have a simple answer, and you mostly learn as you code,
> > difficult
> > > to envision without coding. However simple things like Java services
> can
> > be
> > > mocked with a standard mocking class that you use everywhere.
> > Transactions
> > > are difficult to mock and better left to integration tests I think.
> SECAs
> > > might be mocked by simply passing output to input as chaining methods.
> > Mind
> > > you I am not 100% sure of all of this, coding is the ultimate guide to
> > what
> > > can or cannot be done.
> > >
> > > So I am not suggesting unit tests as a best-practice (even though it
> is).
> > > Instead I suggest it as something that I and others did and got a huge
> > > psychological relief and confidence and comfort from. Swimming in code
> > > without tests is a terrifying business, made more terrifying if the
> code
> > is
> > > bad. Short bursts of red/green/red/green makes you feel good as you
> build
> > > up your logic. And I don't have an exact vision of how to do it,
> because
> > > the details always win.
> > >
> > > I look forward to hearing your thoughts and thank you for enriching
> this
> > > conversation.
> > >
> > > Taher Alkhateeb
> > >
> > > On Tuesday, 19 July 2016, Scott Gray <[hidden email]>
> > wrote:
> > >
> > > > I know I'm late to the party here, but I just want to say that I
> think
> > > > integration tests have far greater value to OFBiz than unit tests.
> > > Mostly
> > > > because we tend to have quite a low number of tests and integration
> > tests
> > > > give us much better coverage per line of test code and the tests are
> > much
> > > > closer to the real world scenarios the application might encounter.
> > > >
> > > > I don't really see how unit tests could be applied to non-framework
> > > testing
> > > > in a useful manner, could you expand on your vision in that regard?
> I
> > > mean
> > > > would we be testing something smaller than a service as the 'unit'?
> > What
> > > > would we mock? Would transaction management still be active? What
> > happens
> > > > when the service calls another service, I guess we mock the response
> > from
> > > > that service (how)?
> > > >
> > > > It just seems a very complicated method to achieve a less thorough
> but
> > > > albeit faster (maybe) test result.
> > > >
> > > > A build, data load and full test run takes 4m 9s on my laptop.
> > Excluding
> > > > build, data load and framework tests: the application level tests
> take
> > > 35s,
> > > > not very expensive IMO.  The data load time can be reduced to
> > practically
> > > > nothing by copying a clean slate database into runtime for each run.
> > > >
> > > > I'm mostly just suggesting we be wary of adding complicated testing
> > > > procedures in the hope of achieving some 'best practice' result which
> > in
> > > > reality will provide minimal benefits.
> > > >
> > > > Regards
> > > > Scott
> > > >
> > > > On 18 July 2016 at 18:57, Taher Alkhateeb <
> [hidden email]
> > > > <javascript:;>>
> > > > wrote:
> > > >
> > > > > Hello Akash,
> > > > >
> > > > > Fantastic, I have a few unit tests almost done to be included in
> the
> > > > start
> > > > > component. I will create a new subtask under OFBIZ-1463 to commit
> the
> > > > tests
> > > > > so you can use them as a reference if you like to.
> > > > >
> > > > > I also recommend that you follow the same directory structure
> between
> > > the
> > > > > test code and production code. So for example:
> > > > >
> > > > > Production code:
> > > > framework/start/src/main/java/org/apache/ofbiz/base/start
> > > > > Test code:
> framework/start/src/test/java/org/apache/ofbiz/base/start
> > > > >
> > > > > The benefit of this hierarchy is that you can access non-public
> > > (package
> > > > > protected) methods for testing. This is in fact exactly what I
> needed
> > > to
> > > > be
> > > > > able to apply some of the tests.
> > > > >
> > > > > Cheers,
> > > > >
> > > > > Taher Alkhateeb
> > > > >
> > > > > On Mon, Jul 18, 2016 at 9:22 AM, Akash Jain <
> > > > [hidden email] <javascript:;>>
> > > > > wrote:
> > > > >
> > > > > > Thanks Taher for nice initiative!
> > > > > >
> > > > > > We are planning to written unit tests to all components under
> > > > OFBIZ-1463
> > > > > >
> > > > > > Thanks and Regards
> > > > > > --
> > > > > > Akash Jain
> > > > > >
> > > > > > On Mon, Jul 18, 2016 at 10:36 AM, Taher Alkhateeb <
> > > > > > [hidden email] <javascript:;>> wrote:
> > > > > >
> > > > > > > Hello Everyone,
> > > > > > >
> > > > > > > In reference to this thread and the Jira OFBIZ-7254, I'm very
> > happy
> > > > to
> > > > > > > announce that OFBiz is now ready for applying unit tests with
> > only
> > > 8
> > > > > new
> > > > > > > lines of code in the build script (r1753143) :)
> > > > > > >
> > > > > > > I recommend we do the following moving forward:
> > > > > > >
> > > > > > > 1- Introduce unit tests as much as we can to all components
> > > > > > > 2- Migrate most of the integration tests we currently have to
> > unit
> > > > > tests
> > > > > > > (since they are designed to do very little integration).
> > > > > > >
> > > > > > > This is a great chance for us to practice real TDD in OFBiz
> > > > > > >
> > > > > > > Cheers,
> > > > > > >
> > > > > > > Taher Alkhateeb
> > > > > > >
> > > > > > > On Mon, Jun 13, 2016 at 7:43 AM, Pranay Pandey <
> > > > > > > [hidden email] <javascript:;>> wrote:
> > > > > > >
> > > > > > > > +1
> > > > > > > >
> > > > > > > > Best regards,
> > > > > > > >
> > > > > > > > Pranay Pandey
> > > > > > > > HotWax Systems
> > > > > > > > http://www.hotwaxsystems.com/
> > > > > > > >
> > > > > > > > On Fri, Jun 10, 2016 at 7:46 PM, Taher Alkhateeb <
> > > > > > > > [hidden email] <javascript:;>
> > > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hello Everyone,
> > > > > > > > >
> > > > > > > > > I was able to get a few tests running and this is very
> > doable.
> > > > But
> > > > > I
> > > > > > > > faced
> > > > > > > > > a big problem in designing the testing framework because of
> > > ANT.
> > > > > > > > >
> > > > > > > > > The problem
> > > > > > > > > ----------------
> > > > > > > > > The way the build scripts are designed in OFBiz are very
> > > > complex. A
> > > > > > > > master
> > > > > > > > > file calls other files which call other files. And in the
> > > middle
> > > > > you
> > > > > > > have
> > > > > > > > > external libraries (ant-contrib) and macros, and variables,
> > and
> > > > > class
> > > > > > > > path
> > > > > > > > > declarations, and and and ....
> > > > > > > > >
> > > > > > > > > I cannot declare the tests programmatically (with JUnit
> test
> > > > > suites)
> > > > > > > > > because this means lower level components would depend on
> > > higher
> > > > > > level
> > > > > > > > > components. So I have to do it in ANT, by navigating this
> > maze
> > > of
> > > > > > build
> > > > > > > > > scripts, and it was a headache for me just to read them,
> let
> > > > alone
> > > > > > > modify
> > > > > > > > > them to create a testing framework.
> > > > > > > > >
> > > > > > > > > Suggested Solution
> > > > > > > > > ------------------------
> > > > > > > > > I suggest to implement the testing framework in Gradle, and
> > > > simply
> > > > > > call
> > > > > > > > it
> > > > > > > > > from within ant. This is a middle solution that sustains
> ant
> > > for
> > > > > now,
> > > > > > > but
> > > > > > > > > can allow us to switch out later.
> > > > > > > > >
> > > > > > > > > This means I will just add one more file called
> build.gradle
> > in
> > > > the
> > > > > > top
> > > > > > > > > level directory, and figure out the business logic for
> > calling
> > > > the
> > > > > > test
> > > > > > > > > suites from that file
> > > > > > > > >
> > > > > > > > > I look forward to your feedback.
> > > > > > > > >
> > > > > > > > > Regards,
> > > > > > > > >
> > > > > > > > > Taher Alkhateeb
> > > > > > > > >
> > > > > > > > > On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <
> > > > > > > > > [hidden email] <javascript:;>>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hi Everyone,
> > > > > > > > > >
> > > > > > > > > > Thank you all for your support, JIRA created in
> > > > > > > > > > https://issues.apache.org/jira/browse/OFBIZ-7254
> > > > > > > > > >
> > > > > > > > > > I will start working on it and try to implement ASAP to
> get
> > > my
> > > > > > focus
> > > > > > > > back
> > > > > > > > > > on refactoring.
> > > > > > > > > >
> > > > > > > > > > Cheers!
> > > > > > > > > >
> > > > > > > > > > Taher Alkhateeb
> > > > > > > > > >
> > > > > > > > > > On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
> > > > > > > > > > [hidden email] <javascript:;>> wrote:
> > > > > > > > > >
> > > > > > > > > >> +1
> > > > > > > > > >>
> > > > > > > > > >> Thanks & Regards
> > > > > > > > > >> --
> > > > > > > > > >> Deepak Dixit
> > > > > > > > > >> www.hotwaxsystems.com
> > > > > > > > > >>
> > > > > > > > > >> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
> > > > > > > > > >> [hidden email] <javascript:;>> wrote:
> > > > > > > > > >>
> > > > > > > > > >> > +1
> > > > > > > > > >> >
> > > > > > > > > >> > Makes perfect sense.
> > > > > > > > > >> >
> > > > > > > > > >> > --
> > > > > > > > > >> > Thanks & Regards,
> > > > > > > > > >> > Mridul Pathak
> > > > > > > > > >> > Senior Manager
> > > > > > > > > >> > HotWax Systems
> > > > > > > > > >> > http://www.hotwaxsystems.com
> > > > > > > > > >> >
> > > > > > > > > >> > > On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
> > > > > > > > > >> [hidden email] <javascript:;>>
> > > > > > > > > >> > wrote:
> > > > > > > > > >> > >
> > > > > > > > > >> > > Hello Everyone,
> > > > > > > > > >> > >
> > > > > > > > > >> > > After refactoring the start component and while
> > starting
> > > > on
> > > > > > the
> > > > > > > > base
> > > > > > > > > >> > > component I realized that the testing framework for
> > > OFBiz
> > > > is
> > > > > > not
> > > > > > > > > good.
> > > > > > > > > >> > You
> > > > > > > > > >> > > cannot do real test driven development or
> > > > red-green-refactor
> > > > > > > with
> > > > > > > > > the
> > > > > > > > > >> > > current setup, hence my proposal to change it. I
> > explain
> > > > > > below:
> > > > > > > > > >> > >
> > > > > > > > > >> > > Problem with current design
> > > > > > > > > >> > > ----------------------------------------
> > > > > > > > > >> > > - What we have right now is not unit tests, it's
> > really
> > > > > > > > integration
> > > > > > > > > >> > tests.
> > > > > > > > > >> > > You have to start the framework, the database, the
> > > service
> > > > > > > engine,
> > > > > > > > > the
> > > > > > > > > >> > > entity engine and pretty much everything.
> > > > > > > > > >> > > - Testing is very slow, because it's an integration
> > test
> > > > as
> > > > > I
> > > > > > > > > >> mentioned
> > > > > > > > > >> > > above. 10 minutes on a good computer!
> > > > > > > > > >> > > - There is zero mocking! We actually have to
> > --load-data
> > > > for
> > > > > > > > things
> > > > > > > > > to
> > > > > > > > > >> > > work. Again, these are integration tests.
> > > > > > > > > >> > > - Too complex: Integration tests by their nature are
> > > > > grabbing
> > > > > > > too
> > > > > > > > > >> much.
> > > > > > > > > >> > > Mind you, I am not objecting to integration tests (I
> > > > > actually
> > > > > > > like
> > > > > > > > > >> them)
> > > > > > > > > >> > > but I am objecting to not having real unit-tests.
> Unit
> > > > tests
> > > > > > > > should
> > > > > > > > > >> all
> > > > > > > > > >> > run
> > > > > > > > > >> > > in a few seconds.
> > > > > > > > > >> > >
> > > > > > > > > >> > > Proposed solution
> > > > > > > > > >> > > --------------------------
> > > > > > > > > >> > > - We keep what is considered real integration tests
> > the
> > > > way
> > > > > > they
> > > > > > > > are
> > > > > > > > > >> > right
> > > > > > > > > >> > > now and keep using them
> > > > > > > > > >> > > - We move what should be unit tests into simple
> JUnit
> > > > > classes,
> > > > > > > and
> > > > > > > > > we
> > > > > > > > > >> do
> > > > > > > > > >> > > not run them using java -jar ofbiz.jar --test, but
> > > instead
> > > > > run
> > > > > > > > them
> > > > > > > > > >> > > directly from the build.xml script, so these files
> are
> > > not
> > > > > > > > > identified
> > > > > > > > > >> in
> > > > > > > > > >> > > any XML document, but are simply called immediately
> > from
> > > > the
> > > > > > > build
> > > > > > > > > >> > scripts.
> > > > > > > > > >> > > - We clearly mark the difference between integration
> > > tests
> > > > > and
> > > > > > > > unit
> > > > > > > > > >> tests
> > > > > > > > > >> > > (inside the source files or in the suite
> > declarations).
> > > > > > > > > >> > > - We change the run-tests target in build.xml to run
> > > both
> > > > > unit
> > > > > > > > tests
> > > > > > > > > >> and
> > > > > > > > > >> > > integration tests.
> > > > > > > > > >> > >
> > > > > > > > > >> > > I intend to heavily refactor the framework and I
> would
> > > > feel
> > > > > > > better
> > > > > > > > > >> about
> > > > > > > > > >> > > introducing this change while refactoring. What do
> you
> > > > guys
> > > > > > > think?
> > > > > > > > > >> Ideas?
> > > > > > > > > >> > > Suggestions? Approvals and thumbs up?
> > > > > > > > > >> > >
> > > > > > > > > >> > > Regards,
> > > > > > > > > >> > >
> > > > > > > > > >> > > Taher Alkhateeb
> > > > > > > > > >> >
> > > > > > > > > >> >
> > > > > > > > > >>
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

[taher]

Hi Scott,

Well spoken thank you. Okay may I suggest that for any such work we will
discuss it here see its Merit and if it makes sense then we take it in. It
is a little early to discuss it right now because we did not yet go to the
higher-level components. Once we do I'll be sure to have a conversation
about this in here and would appreciate your input to it.

Regards,

Taher Alkhateeb

On Jul 19, 2016 9:00 AM, "Scott Gray" <[hidden email]> wrote:

> Yeah I'm sure unit tests probably worked well for the start component given
> it is the lowest level component in the system and closest to a basic java
> app.  I just think the value proposition might decrease the further up the
> stack we move with them.  I'm not against unit tests when they prove
> useful, but further up the stack I think we should prove the case for them
> before doing much work to support the mocking that will be required to keep
> them inline with 'best practices'.
>
> In OFBiz "bad or unwanted" tends to come mostly in the way of increasing
> complexity from features that require more effort to maintain than the
> value they provide to the community.  I think there's a chance attempting
> to mock service tests could fall into that bucket.  I could be wrong, but I
> think it's worth looking into before we declare that unit tests are the
> best form of testing for OFBiz.
>
> Regards
> Scott
>
> On 19 July 2016 at 17:37, Taher Alkhateeb <[hidden email]>
> wrote:
>
> > Hi Scott,
> >
> > Thank you for the feedback. To be focused exactly on integration vs
> unit, I
> > already mentioned above that at least for me the main objective is to
> > confidently and quickly run the tests in short bursts of red-green
> > refactor. This allows me to refactor code without waiting in front of the
> > screen in between test cycles thus giving me immediate feedback on any
> > errors I made. Perhaps my intro was too long so this is the squeezed out
> > version of it.
> >
> > I already had one round of testing in the start component which was much
> > faster that way and had an immediate impact. Oh and by the way, you
> cannot
> > test the start component with integration tests for example unless you do
> > it from an external component which cannot access package protected
> items.
> >
> > This style of coding is applicable I think to any software project
> > inclusive of OFBiz. Maybe in certain components more than others or
> certain
> > areas more than others. But I can't see how it could be bad or unwanted.
> >
> > Taher Alkhateeb
> >
> > On Jul 19, 2016 8:18 AM, "Scott Gray" <[hidden email]>
> > wrote:
> >
> > > Thanks Ron and Taher for your responses, I appreciate them but I don't
> > hear
> > > much in the conversation that speaks to the value of unit tests over
> > > integration tests.  Most of the thoughts shared speaks more to the
> value
> > of
> > > tests rather than differences in the type of tests.
> > >
> > > Speed: At an application level we have ~685 tests that run in 35
> seconds
> > > (excluding build and data load).  Another point is that there isn't
> much
> > > reason why tests can't be run in parallel rather than sequentially.
> > > TDD: Integration tests in OFBiz are as simple if not simpler to write
> > than
> > > unit tests and just as useful for TDD.
> > > I'm not sure if I missed any other points raised regarding integration
> > vs.
> > > unit tests?
> > >
> > > I'm not looking to start a big long debate on the topic, I just wanted
> to
> > > speak out that someone out there (me) doesn't think unit tests are the
> > best
> > > solution for testing OFBiz applications.
> > >
> > > Regards
> > > Scott
> > >
> > > On 19 July 2016 at 16:52, Taher Alkhateeb <[hidden email]>
> > > wrote:
> > >
> > > > Hi Scott,
> > > >
> > > > Thank you for your input. Your ideas are thought provoking and
> > enriching
> > > to
> > > > the conversation.
> > > >
> > > > The way I look at it, the general rule is usually many unit tests,
> less
> > > > integration tests, lesser functional tests. So we are not excluding
> any
> > > > types of test, all of them are important in certain areas for certain
> > > > purposes with certain quantity. Usually integration tests are less
> > > because
> > > > as you said they just grab more. I like the picture below as a
> general
> > > > guideline
> > > >
> > > > http://i.stack.imgur.com/fjQvQ.png
> > > >
> > > > As you already mentioned unit tests are useful for the framework. I
> > > > discovered errors in code that I wrote which I was very very careful
> > > with.
> > > > I immediately learned the lesson that humans are not designed to
> code,
> > > and
> > > > TDD gives you confidence as you build your code with those short
> 30-60
> > > > second red-green refactors. I feel much much safer and more confident
> > > > writing code that way, the test also documents how to use the api,
> > > > refactoring and feature change becomes less terrifying. Also messy
> code
> > > is
> > > > usually not test friendly thus refactoring and unit tests go
> > hand-in-hand
> > > > for improving the code base.
> > > >
> > > > Also, I am sorry to say that the framework code is rather messy and
> > > brittle
> > > > in many places. I think you probably encountered this yourself. The
> > same
> > > is
> > > > said for the applications. Now If we start refactoring without unit
> > tests
> > > > then we are back to scary business. So much can go wrong so fast and
> > > break
> > > > things you never expected to break. The framework with all
> applications
> > > > require heavy refactoring of things like massive ugly methods,
> > sandwiched
> > > > logic, heavy shared mutable state, hidden dependencies, poor
> > > > interfaces, and much more. Every time I touch something I get shocked
> > at
> > > > how bad it looks, spaghetti logic to no end. If you refactor with
> > > > integration tests instead of unit tests then you will come down to a
> > > > screeching halt as you wait between these test cycles. My computer
> > > actually
> > > > takes 10 minutes or more for a full clean load data and testing.
> > > >
> > > > Now talking about mocking you raise some interesting points, what can
> > you
> > > > mock vs utilizing integration tests? This is an excellent question
> that
> > > > does not have a simple answer, and you mostly learn as you code,
> > > difficult
> > > > to envision without coding. However simple things like Java services
> > can
> > > be
> > > > mocked with a standard mocking class that you use everywhere.
> > > Transactions
> > > > are difficult to mock and better left to integration tests I think.
> > SECAs
> > > > might be mocked by simply passing output to input as chaining
> methods.
> > > Mind
> > > > you I am not 100% sure of all of this, coding is the ultimate guide
> to
> > > what
> > > > can or cannot be done.
> > > >
> > > > So I am not suggesting unit tests as a best-practice (even though it
> > is).
> > > > Instead I suggest it as something that I and others did and got a
> huge
> > > > psychological relief and confidence and comfort from. Swimming in
> code
> > > > without tests is a terrifying business, made more terrifying if the
> > code
> > > is
> > > > bad. Short bursts of red/green/red/green makes you feel good as you
> > build
> > > > up your logic. And I don't have an exact vision of how to do it,
> > because
> > > > the details always win.
> > > >
> > > > I look forward to hearing your thoughts and thank you for enriching
> > this
> > > > conversation.
> > > >
> > > > Taher Alkhateeb
> > > >
> > > > On Tuesday, 19 July 2016, Scott Gray <[hidden email]>
> > > wrote:
> > > >
> > > > > I know I'm late to the party here, but I just want to say that I
> > think
> > > > > integration tests have far greater value to OFBiz than unit tests.
> > > > Mostly
> > > > > because we tend to have quite a low number of tests and integration
> > > tests
> > > > > give us much better coverage per line of test code and the tests
> are
> > > much
> > > > > closer to the real world scenarios the application might encounter.
> > > > >
> > > > > I don't really see how unit tests could be applied to non-framework
> > > > testing
> > > > > in a useful manner, could you expand on your vision in that regard?
> > I
> > > > mean
> > > > > would we be testing something smaller than a service as the 'unit'?
> > > What
> > > > > would we mock? Would transaction management still be active? What
> > > happens
> > > > > when the service calls another service, I guess we mock the
> response
> > > from
> > > > > that service (how)?
> > > > >
> > > > > It just seems a very complicated method to achieve a less thorough
> > but
> > > > > albeit faster (maybe) test result.
> > > > >
> > > > > A build, data load and full test run takes 4m 9s on my laptop.
> > > Excluding
> > > > > build, data load and framework tests: the application level tests
> > take
> > > > 35s,
> > > > > not very expensive IMO.  The data load time can be reduced to
> > > practically
> > > > > nothing by copying a clean slate database into runtime for each
> run.
> > > > >
> > > > > I'm mostly just suggesting we be wary of adding complicated testing
> > > > > procedures in the hope of achieving some 'best practice' result
> which
> > > in
> > > > > reality will provide minimal benefits.
> > > > >
> > > > > Regards
> > > > > Scott
> > > > >
> > > > > On 18 July 2016 at 18:57, Taher Alkhateeb <
> > [hidden email]
> > > > > <javascript:;>>
> > > > > wrote:
> > > > >
> > > > > > Hello Akash,
> > > > > >
> > > > > > Fantastic, I have a few unit tests almost done to be included in
> > the
> > > > > start
> > > > > > component. I will create a new subtask under OFBIZ-1463 to commit
> > the
> > > > > tests
> > > > > > so you can use them as a reference if you like to.
> > > > > >
> > > > > > I also recommend that you follow the same directory structure
> > between
> > > > the
> > > > > > test code and production code. So for example:
> > > > > >
> > > > > > Production code:
> > > > > framework/start/src/main/java/org/apache/ofbiz/base/start
> > > > > > Test code:
> > framework/start/src/test/java/org/apache/ofbiz/base/start
> > > > > >
> > > > > > The benefit of this hierarchy is that you can access non-public
> > > > (package
> > > > > > protected) methods for testing. This is in fact exactly what I
> > needed
> > > > to
> > > > > be
> > > > > > able to apply some of the tests.
> > > > > >
> > > > > > Cheers,
> > > > > >
> > > > > > Taher Alkhateeb
> > > > > >
> > > > > > On Mon, Jul 18, 2016 at 9:22 AM, Akash Jain <
> > > > > [hidden email] <javascript:;>>
> > > > > > wrote:
> > > > > >
> > > > > > > Thanks Taher for nice initiative!
> > > > > > >
> > > > > > > We are planning to written unit tests to all components under
> > > > > OFBIZ-1463
> > > > > > >
> > > > > > > Thanks and Regards
> > > > > > > --
> > > > > > > Akash Jain
> > > > > > >
> > > > > > > On Mon, Jul 18, 2016 at 10:36 AM, Taher Alkhateeb <
> > > > > > > [hidden email] <javascript:;>> wrote:
> > > > > > >
> > > > > > > > Hello Everyone,
> > > > > > > >
> > > > > > > > In reference to this thread and the Jira OFBIZ-7254, I'm very
> > > happy
> > > > > to
> > > > > > > > announce that OFBiz is now ready for applying unit tests with
> > > only
> > > > 8
> > > > > > new
> > > > > > > > lines of code in the build script (r1753143) :)
> > > > > > > >
> > > > > > > > I recommend we do the following moving forward:
> > > > > > > >
> > > > > > > > 1- Introduce unit tests as much as we can to all components
> > > > > > > > 2- Migrate most of the integration tests we currently have to
> > > unit
> > > > > > tests
> > > > > > > > (since they are designed to do very little integration).
> > > > > > > >
> > > > > > > > This is a great chance for us to practice real TDD in OFBiz
> > > > > > > >
> > > > > > > > Cheers,
> > > > > > > >
> > > > > > > > Taher Alkhateeb
> > > > > > > >
> > > > > > > > On Mon, Jun 13, 2016 at 7:43 AM, Pranay Pandey <
> > > > > > > > [hidden email] <javascript:;>> wrote:
> > > > > > > >
> > > > > > > > > +1
> > > > > > > > >
> > > > > > > > > Best regards,
> > > > > > > > >
> > > > > > > > > Pranay Pandey
> > > > > > > > > HotWax Systems
> > > > > > > > > http://www.hotwaxsystems.com/
> > > > > > > > >
> > > > > > > > > On Fri, Jun 10, 2016 at 7:46 PM, Taher Alkhateeb <
> > > > > > > > > [hidden email] <javascript:;>
> > > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hello Everyone,
> > > > > > > > > >
> > > > > > > > > > I was able to get a few tests running and this is very
> > > doable.
> > > > > But
> > > > > > I
> > > > > > > > > faced
> > > > > > > > > > a big problem in designing the testing framework because
> of
> > > > ANT.
> > > > > > > > > >
> > > > > > > > > > The problem
> > > > > > > > > > ----------------
> > > > > > > > > > The way the build scripts are designed in OFBiz are very
> > > > > complex. A
> > > > > > > > > master
> > > > > > > > > > file calls other files which call other files. And in the
> > > > middle
> > > > > > you
> > > > > > > > have
> > > > > > > > > > external libraries (ant-contrib) and macros, and
> variables,
> > > and
> > > > > > class
> > > > > > > > > path
> > > > > > > > > > declarations, and and and ....
> > > > > > > > > >
> > > > > > > > > > I cannot declare the tests programmatically (with JUnit
> > test
> > > > > > suites)
> > > > > > > > > > because this means lower level components would depend on
> > > > higher
> > > > > > > level
> > > > > > > > > > components. So I have to do it in ANT, by navigating this
> > > maze
> > > > of
> > > > > > > build
> > > > > > > > > > scripts, and it was a headache for me just to read them,
> > let
> > > > > alone
> > > > > > > > modify
> > > > > > > > > > them to create a testing framework.
> > > > > > > > > >
> > > > > > > > > > Suggested Solution
> > > > > > > > > > ------------------------
> > > > > > > > > > I suggest to implement the testing framework in Gradle,
> and
> > > > > simply
> > > > > > > call
> > > > > > > > > it
> > > > > > > > > > from within ant. This is a middle solution that sustains
> > ant
> > > > for
> > > > > > now,
> > > > > > > > but
> > > > > > > > > > can allow us to switch out later.
> > > > > > > > > >
> > > > > > > > > > This means I will just add one more file called
> > build.gradle
> > > in
> > > > > the
> > > > > > > top
> > > > > > > > > > level directory, and figure out the business logic for
> > > calling
> > > > > the
> > > > > > > test
> > > > > > > > > > suites from that file
> > > > > > > > > >
> > > > > > > > > > I look forward to your feedback.
> > > > > > > > > >
> > > > > > > > > > Regards,
> > > > > > > > > >
> > > > > > > > > > Taher Alkhateeb
> > > > > > > > > >
> > > > > > > > > > On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <
> > > > > > > > > > [hidden email] <javascript:;>>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Hi Everyone,
> > > > > > > > > > >
> > > > > > > > > > > Thank you all for your support, JIRA created in
> > > > > > > > > > > https://issues.apache.org/jira/browse/OFBIZ-7254
> > > > > > > > > > >
> > > > > > > > > > > I will start working on it and try to implement ASAP to
> > get
> > > > my
> > > > > > > focus
> > > > > > > > > back
> > > > > > > > > > > on refactoring.
> > > > > > > > > > >
> > > > > > > > > > > Cheers!
> > > > > > > > > > >
> > > > > > > > > > > Taher Alkhateeb
> > > > > > > > > > >
> > > > > > > > > > > On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
> > > > > > > > > > > [hidden email] <javascript:;>> wrote:
> > > > > > > > > > >
> > > > > > > > > > >> +1
> > > > > > > > > > >>
> > > > > > > > > > >> Thanks & Regards
> > > > > > > > > > >> --
> > > > > > > > > > >> Deepak Dixit
> > > > > > > > > > >> www.hotwaxsystems.com
> > > > > > > > > > >>
> > > > > > > > > > >> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
> > > > > > > > > > >> [hidden email] <javascript:;>>
> wrote:
> > > > > > > > > > >>
> > > > > > > > > > >> > +1
> > > > > > > > > > >> >
> > > > > > > > > > >> > Makes perfect sense.
> > > > > > > > > > >> >
> > > > > > > > > > >> > --
> > > > > > > > > > >> > Thanks & Regards,
> > > > > > > > > > >> > Mridul Pathak
> > > > > > > > > > >> > Senior Manager
> > > > > > > > > > >> > HotWax Systems
> > > > > > > > > > >> > http://www.hotwaxsystems.com
> > > > > > > > > > >> >
> > > > > > > > > > >> > > On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
> > > > > > > > > > >> [hidden email] <javascript:;>>
> > > > > > > > > > >> > wrote:
> > > > > > > > > > >> > >
> > > > > > > > > > >> > > Hello Everyone,
> > > > > > > > > > >> > >
> > > > > > > > > > >> > > After refactoring the start component and while
> > > starting
> > > > > on
> > > > > > > the
> > > > > > > > > base
> > > > > > > > > > >> > > component I realized that the testing framework
> for
> > > > OFBiz
> > > > > is
> > > > > > > not
> > > > > > > > > > good.
> > > > > > > > > > >> > You
> > > > > > > > > > >> > > cannot do real test driven development or
> > > > > red-green-refactor
> > > > > > > > with
> > > > > > > > > > the
> > > > > > > > > > >> > > current setup, hence my proposal to change it. I
> > > explain
> > > > > > > below:
> > > > > > > > > > >> > >
> > > > > > > > > > >> > > Problem with current design
> > > > > > > > > > >> > > ----------------------------------------
> > > > > > > > > > >> > > - What we have right now is not unit tests, it's
> > > really
> > > > > > > > > integration
> > > > > > > > > > >> > tests.
> > > > > > > > > > >> > > You have to start the framework, the database, the
> > > > service
> > > > > > > > engine,
> > > > > > > > > > the
> > > > > > > > > > >> > > entity engine and pretty much everything.
> > > > > > > > > > >> > > - Testing is very slow, because it's an
> integration
> > > test
> > > > > as
> > > > > > I
> > > > > > > > > > >> mentioned
> > > > > > > > > > >> > > above. 10 minutes on a good computer!
> > > > > > > > > > >> > > - There is zero mocking! We actually have to
> > > --load-data
> > > > > for
> > > > > > > > > things
> > > > > > > > > > to
> > > > > > > > > > >> > > work. Again, these are integration tests.
> > > > > > > > > > >> > > - Too complex: Integration tests by their nature
> are
> > > > > > grabbing
> > > > > > > > too
> > > > > > > > > > >> much.
> > > > > > > > > > >> > > Mind you, I am not objecting to integration tests
> (I
> > > > > > actually
> > > > > > > > like
> > > > > > > > > > >> them)
> > > > > > > > > > >> > > but I am objecting to not having real unit-tests.
> > Unit
> > > > > tests
> > > > > > > > > should
> > > > > > > > > > >> all
> > > > > > > > > > >> > run
> > > > > > > > > > >> > > in a few seconds.
> > > > > > > > > > >> > >
> > > > > > > > > > >> > > Proposed solution
> > > > > > > > > > >> > > --------------------------
> > > > > > > > > > >> > > - We keep what is considered real integration
> tests
> > > the
> > > > > way
> > > > > > > they
> > > > > > > > > are
> > > > > > > > > > >> > right
> > > > > > > > > > >> > > now and keep using them
> > > > > > > > > > >> > > - We move what should be unit tests into simple
> > JUnit
> > > > > > classes,
> > > > > > > > and
> > > > > > > > > > we
> > > > > > > > > > >> do
> > > > > > > > > > >> > > not run them using java -jar ofbiz.jar --test, but
> > > > instead
> > > > > > run
> > > > > > > > > them
> > > > > > > > > > >> > > directly from the build.xml script, so these files
> > are
> > > > not
> > > > > > > > > > identified
> > > > > > > > > > >> in
> > > > > > > > > > >> > > any XML document, but are simply called
> immediately
> > > from
> > > > > the
> > > > > > > > build
> > > > > > > > > > >> > scripts.
> > > > > > > > > > >> > > - We clearly mark the difference between
> integration
> > > > tests
> > > > > > and
> > > > > > > > > unit
> > > > > > > > > > >> tests
> > > > > > > > > > >> > > (inside the source files or in the suite
> > > declarations).
> > > > > > > > > > >> > > - We change the run-tests target in build.xml to
> run
> > > > both
> > > > > > unit
> > > > > > > > > tests
> > > > > > > > > > >> and
> > > > > > > > > > >> > > integration tests.
> > > > > > > > > > >> > >
> > > > > > > > > > >> > > I intend to heavily refactor the framework and I
> > would
> > > > > feel
> > > > > > > > better
> > > > > > > > > > >> about
> > > > > > > > > > >> > > introducing this change while refactoring. What do
> > you
> > > > > guys
> > > > > > > > think?
> > > > > > > > > > >> Ideas?
> > > > > > > > > > >> > > Suggestions? Approvals and thumbs up?
> > > > > > > > > > >> > >
> > > > > > > > > > >> > > Regards,
> > > > > > > > > > >> > >
> > > > > > > > > > >> > > Taher Alkhateeb
> > > > > > > > > > >> >
> > > > > > > > > > >> >
> > > > > > > > > > >>
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

[Pierre Smits]

As with anything, the law of diminishing returns also applies to OFBiz and
tests. This is not true for unit tests and system integration tests, but
also for user acceptance and performance tests.

Nevertheless, the work done up to now is a good start and - I feel
confident - appreciated. And unit tests are certainly valuable in the
framework stack. How it will be for functions (regarding components in
application and special purpose stack) needs to be addressed when we reach
that bridge.


Best regards,

Pierre Smits

ORRTIZ.COM <http://www.orrtiz.com>
OFBiz based solutions & services

OFBiz Extensions Marketplace
http://oem.ofbizci.net/oci-2/

On Tue, Jul 19, 2016 at 8:22 AM, Taher Alkhateeb <[hidden email]
> wrote:

> Hi Scott,
>
> Well spoken thank you. Okay may I suggest that for any such work we will
> discuss it here see its Merit and if it makes sense then we take it in. It
> is a little early to discuss it right now because we did not yet go to the
> higher-level components. Once we do I'll be sure to have a conversation
> about this in here and would appreciate your input to it.
>
> Regards,
>
> Taher Alkhateeb
>
> On Jul 19, 2016 9:00 AM, "Scott Gray" <[hidden email]>
> wrote:
>
> > Yeah I'm sure unit tests probably worked well for the start component
> given
> > it is the lowest level component in the system and closest to a basic
> java
> > app.  I just think the value proposition might decrease the further up
> the
> > stack we move with them.  I'm not against unit tests when they prove
> > useful, but further up the stack I think we should prove the case for
> them
> > before doing much work to support the mocking that will be required to
> keep
> > them inline with 'best practices'.
> >
> > In OFBiz "bad or unwanted" tends to come mostly in the way of increasing
> > complexity from features that require more effort to maintain than the
> > value they provide to the community.  I think there's a chance attempting
> > to mock service tests could fall into that bucket.  I could be wrong,
> but I
> > think it's worth looking into before we declare that unit tests are the
> > best form of testing for OFBiz.
> >
> > Regards
> > Scott
> >
> > On 19 July 2016 at 17:37, Taher Alkhateeb <[hidden email]>
> > wrote:
> >
> > > Hi Scott,
> > >
> > > Thank you for the feedback. To be focused exactly on integration vs
> > unit, I
> > > already mentioned above that at least for me the main objective is to
> > > confidently and quickly run the tests in short bursts of red-green
> > > refactor. This allows me to refactor code without waiting in front of
> the
> > > screen in between test cycles thus giving me immediate feedback on any
> > > errors I made. Perhaps my intro was too long so this is the squeezed
> out
> > > version of it.
> > >
> > > I already had one round of testing in the start component which was
> much
> > > faster that way and had an immediate impact. Oh and by the way, you
> > cannot
> > > test the start component with integration tests for example unless you
> do
> > > it from an external component which cannot access package protected
> > items.
> > >
> > > This style of coding is applicable I think to any software project
> > > inclusive of OFBiz. Maybe in certain components more than others or
> > certain
> > > areas more than others. But I can't see how it could be bad or
> unwanted.
> > >
> > > Taher Alkhateeb
> > >
> > > On Jul 19, 2016 8:18 AM, "Scott Gray" <[hidden email]>
> > > wrote:
> > >
> > > > Thanks Ron and Taher for your responses, I appreciate them but I
> don't
> > > hear
> > > > much in the conversation that speaks to the value of unit tests over
> > > > integration tests.  Most of the thoughts shared speaks more to the
> > value
> > > of
> > > > tests rather than differences in the type of tests.
> > > >
> > > > Speed: At an application level we have ~685 tests that run in 35
> > seconds
> > > > (excluding build and data load).  Another point is that there isn't
> > much
> > > > reason why tests can't be run in parallel rather than sequentially.
> > > > TDD: Integration tests in OFBiz are as simple if not simpler to write
> > > than
> > > > unit tests and just as useful for TDD.
> > > > I'm not sure if I missed any other points raised regarding
> integration
> > > vs.
> > > > unit tests?
> > > >
> > > > I'm not looking to start a big long debate on the topic, I just
> wanted
> > to
> > > > speak out that someone out there (me) doesn't think unit tests are
> the
> > > best
> > > > solution for testing OFBiz applications.
> > > >
> > > > Regards
> > > > Scott
> > > >
> > > > On 19 July 2016 at 16:52, Taher Alkhateeb <
> [hidden email]>
> > > > wrote:
> > > >
> > > > > Hi Scott,
> > > > >
> > > > > Thank you for your input. Your ideas are thought provoking and
> > > enriching
> > > > to
> > > > > the conversation.
> > > > >
> > > > > The way I look at it, the general rule is usually many unit tests,
> > less
> > > > > integration tests, lesser functional tests. So we are not excluding
> > any
> > > > > types of test, all of them are important in certain areas for
> certain
> > > > > purposes with certain quantity. Usually integration tests are less
> > > > because
> > > > > as you said they just grab more. I like the picture below as a
> > general
> > > > > guideline
> > > > >
> > > > > http://i.stack.imgur.com/fjQvQ.png
> > > > >
> > > > > As you already mentioned unit tests are useful for the framework. I
> > > > > discovered errors in code that I wrote which I was very very
> careful
> > > > with.
> > > > > I immediately learned the lesson that humans are not designed to
> > code,
> > > > and
> > > > > TDD gives you confidence as you build your code with those short
> > 30-60
> > > > > second red-green refactors. I feel much much safer and more
> confident
> > > > > writing code that way, the test also documents how to use the api,
> > > > > refactoring and feature change becomes less terrifying. Also messy
> > code
> > > > is
> > > > > usually not test friendly thus refactoring and unit tests go
> > > hand-in-hand
> > > > > for improving the code base.
> > > > >
> > > > > Also, I am sorry to say that the framework code is rather messy and
> > > > brittle
> > > > > in many places. I think you probably encountered this yourself. The
> > > same
> > > > is
> > > > > said for the applications. Now If we start refactoring without unit
> > > tests
> > > > > then we are back to scary business. So much can go wrong so fast
> and
> > > > break
> > > > > things you never expected to break. The framework with all
> > applications
> > > > > require heavy refactoring of things like massive ugly methods,
> > > sandwiched
> > > > > logic, heavy shared mutable state, hidden dependencies, poor
> > > > > interfaces, and much more. Every time I touch something I get
> shocked
> > > at
> > > > > how bad it looks, spaghetti logic to no end. If you refactor with
> > > > > integration tests instead of unit tests then you will come down to
> a
> > > > > screeching halt as you wait between these test cycles. My computer
> > > > actually
> > > > > takes 10 minutes or more for a full clean load data and testing.
> > > > >
> > > > > Now talking about mocking you raise some interesting points, what
> can
> > > you
> > > > > mock vs utilizing integration tests? This is an excellent question
> > that
> > > > > does not have a simple answer, and you mostly learn as you code,
> > > > difficult
> > > > > to envision without coding. However simple things like Java
> services
> > > can
> > > > be
> > > > > mocked with a standard mocking class that you use everywhere.
> > > > Transactions
> > > > > are difficult to mock and better left to integration tests I think.
> > > SECAs
> > > > > might be mocked by simply passing output to input as chaining
> > methods.
> > > > Mind
> > > > > you I am not 100% sure of all of this, coding is the ultimate guide
> > to
> > > > what
> > > > > can or cannot be done.
> > > > >
> > > > > So I am not suggesting unit tests as a best-practice (even though
> it
> > > is).
> > > > > Instead I suggest it as something that I and others did and got a
> > huge
> > > > > psychological relief and confidence and comfort from. Swimming in
> > code
> > > > > without tests is a terrifying business, made more terrifying if the
> > > code
> > > > is
> > > > > bad. Short bursts of red/green/red/green makes you feel good as you
> > > build
> > > > > up your logic. And I don't have an exact vision of how to do it,
> > > because
> > > > > the details always win.
> > > > >
> > > > > I look forward to hearing your thoughts and thank you for enriching
> > > this
> > > > > conversation.
> > > > >
> > > > > Taher Alkhateeb
> > > > >
> > > > > On Tuesday, 19 July 2016, Scott Gray <[hidden email]
> >
> > > > wrote:
> > > > >
> > > > > > I know I'm late to the party here, but I just want to say that I
> > > think
> > > > > > integration tests have far greater value to OFBiz than unit
> tests.
> > > > > Mostly
> > > > > > because we tend to have quite a low number of tests and
> integration
> > > > tests
> > > > > > give us much better coverage per line of test code and the tests
> > are
> > > > much
> > > > > > closer to the real world scenarios the application might
> encounter.
> > > > > >
> > > > > > I don't really see how unit tests could be applied to
> non-framework
> > > > > testing
> > > > > > in a useful manner, could you expand on your vision in that
> regard?
> > > I
> > > > > mean
> > > > > > would we be testing something smaller than a service as the
> 'unit'?
> > > > What
> > > > > > would we mock? Would transaction management still be active? What
> > > > happens
> > > > > > when the service calls another service, I guess we mock the
> > response
> > > > from
> > > > > > that service (how)?
> > > > > >
> > > > > > It just seems a very complicated method to achieve a less
> thorough
> > > but
> > > > > > albeit faster (maybe) test result.
> > > > > >
> > > > > > A build, data load and full test run takes 4m 9s on my laptop.
> > > > Excluding
> > > > > > build, data load and framework tests: the application level tests
> > > take
> > > > > 35s,
> > > > > > not very expensive IMO.  The data load time can be reduced to
> > > > practically
> > > > > > nothing by copying a clean slate database into runtime for each
> > run.
> > > > > >
> > > > > > I'm mostly just suggesting we be wary of adding complicated
> testing
> > > > > > procedures in the hope of achieving some 'best practice' result
> > which
> > > > in
> > > > > > reality will provide minimal benefits.
> > > > > >
> > > > > > Regards
> > > > > > Scott
> > > > > >
> > > > > > On 18 July 2016 at 18:57, Taher Alkhateeb <
> > > [hidden email]
> > > > > > <javascript:;>>
> > > > > > wrote:
> > > > > >
> > > > > > > Hello Akash,
> > > > > > >
> > > > > > > Fantastic, I have a few unit tests almost done to be included
> in
> > > the
> > > > > > start
> > > > > > > component. I will create a new subtask under OFBIZ-1463 to
> commit
> > > the
> > > > > > tests
> > > > > > > so you can use them as a reference if you like to.
> > > > > > >
> > > > > > > I also recommend that you follow the same directory structure
> > > between
> > > > > the
> > > > > > > test code and production code. So for example:
> > > > > > >
> > > > > > > Production code:
> > > > > > framework/start/src/main/java/org/apache/ofbiz/base/start
> > > > > > > Test code:
> > > framework/start/src/test/java/org/apache/ofbiz/base/start
> > > > > > >
> > > > > > > The benefit of this hierarchy is that you can access non-public
> > > > > (package
> > > > > > > protected) methods for testing. This is in fact exactly what I
> > > needed
> > > > > to
> > > > > > be
> > > > > > > able to apply some of the tests.
> > > > > > >
> > > > > > > Cheers,
> > > > > > >
> > > > > > > Taher Alkhateeb
> > > > > > >
> > > > > > > On Mon, Jul 18, 2016 at 9:22 AM, Akash Jain <
> > > > > > [hidden email] <javascript:;>>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Thanks Taher for nice initiative!
> > > > > > > >
> > > > > > > > We are planning to written unit tests to all components under
> > > > > > OFBIZ-1463
> > > > > > > >
> > > > > > > > Thanks and Regards
> > > > > > > > --
> > > > > > > > Akash Jain
> > > > > > > >
> > > > > > > > On Mon, Jul 18, 2016 at 10:36 AM, Taher Alkhateeb <
> > > > > > > > [hidden email] <javascript:;>> wrote:
> > > > > > > >
> > > > > > > > > Hello Everyone,
> > > > > > > > >
> > > > > > > > > In reference to this thread and the Jira OFBIZ-7254, I'm
> very
> > > > happy
> > > > > > to
> > > > > > > > > announce that OFBiz is now ready for applying unit tests
> with
> > > > only
> > > > > 8
> > > > > > > new
> > > > > > > > > lines of code in the build script (r1753143) :)
> > > > > > > > >
> > > > > > > > > I recommend we do the following moving forward:
> > > > > > > > >
> > > > > > > > > 1- Introduce unit tests as much as we can to all components
> > > > > > > > > 2- Migrate most of the integration tests we currently have
> to
> > > > unit
> > > > > > > tests
> > > > > > > > > (since they are designed to do very little integration).
> > > > > > > > >
> > > > > > > > > This is a great chance for us to practice real TDD in OFBiz
> > > > > > > > >
> > > > > > > > > Cheers,
> > > > > > > > >
> > > > > > > > > Taher Alkhateeb
> > > > > > > > >
> > > > > > > > > On Mon, Jun 13, 2016 at 7:43 AM, Pranay Pandey <
> > > > > > > > > [hidden email] <javascript:;>> wrote:
> > > > > > > > >
> > > > > > > > > > +1
> > > > > > > > > >
> > > > > > > > > > Best regards,
> > > > > > > > > >
> > > > > > > > > > Pranay Pandey
> > > > > > > > > > HotWax Systems
> > > > > > > > > > http://www.hotwaxsystems.com/
> > > > > > > > > >
> > > > > > > > > > On Fri, Jun 10, 2016 at 7:46 PM, Taher Alkhateeb <
> > > > > > > > > > [hidden email] <javascript:;>
> > > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Hello Everyone,
> > > > > > > > > > >
> > > > > > > > > > > I was able to get a few tests running and this is very
> > > > doable.
> > > > > > But
> > > > > > > I
> > > > > > > > > > faced
> > > > > > > > > > > a big problem in designing the testing framework
> because
> > of
> > > > > ANT.
> > > > > > > > > > >
> > > > > > > > > > > The problem
> > > > > > > > > > > ----------------
> > > > > > > > > > > The way the build scripts are designed in OFBiz are
> very
> > > > > > complex. A
> > > > > > > > > > master
> > > > > > > > > > > file calls other files which call other files. And in
> the
> > > > > middle
> > > > > > > you
> > > > > > > > > have
> > > > > > > > > > > external libraries (ant-contrib) and macros, and
> > variables,
> > > > and
> > > > > > > class
> > > > > > > > > > path
> > > > > > > > > > > declarations, and and and ....
> > > > > > > > > > >
> > > > > > > > > > > I cannot declare the tests programmatically (with JUnit
> > > test
> > > > > > > suites)
> > > > > > > > > > > because this means lower level components would depend
> on
> > > > > higher
> > > > > > > > level
> > > > > > > > > > > components. So I have to do it in ANT, by navigating
> this
> > > > maze
> > > > > of
> > > > > > > > build
> > > > > > > > > > > scripts, and it was a headache for me just to read
> them,
> > > let
> > > > > > alone
> > > > > > > > > modify
> > > > > > > > > > > them to create a testing framework.
> > > > > > > > > > >
> > > > > > > > > > > Suggested Solution
> > > > > > > > > > > ------------------------
> > > > > > > > > > > I suggest to implement the testing framework in Gradle,
> > and
> > > > > > simply
> > > > > > > > call
> > > > > > > > > > it
> > > > > > > > > > > from within ant. This is a middle solution that
> sustains
> > > ant
> > > > > for
> > > > > > > now,
> > > > > > > > > but
> > > > > > > > > > > can allow us to switch out later.
> > > > > > > > > > >
> > > > > > > > > > > This means I will just add one more file called
> > > build.gradle
> > > > in
> > > > > > the
> > > > > > > > top
> > > > > > > > > > > level directory, and figure out the business logic for
> > > > calling
> > > > > > the
> > > > > > > > test
> > > > > > > > > > > suites from that file
> > > > > > > > > > >
> > > > > > > > > > > I look forward to your feedback.
> > > > > > > > > > >
> > > > > > > > > > > Regards,
> > > > > > > > > > >
> > > > > > > > > > > Taher Alkhateeb
> > > > > > > > > > >
> > > > > > > > > > > On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <
> > > > > > > > > > > [hidden email] <javascript:;>>
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Hi Everyone,
> > > > > > > > > > > >
> > > > > > > > > > > > Thank you all for your support, JIRA created in
> > > > > > > > > > > > https://issues.apache.org/jira/browse/OFBIZ-7254
> > > > > > > > > > > >
> > > > > > > > > > > > I will start working on it and try to implement ASAP
> to
> > > get
> > > > > my
> > > > > > > > focus
> > > > > > > > > > back
> > > > > > > > > > > > on refactoring.
> > > > > > > > > > > >
> > > > > > > > > > > > Cheers!
> > > > > > > > > > > >
> > > > > > > > > > > > Taher Alkhateeb
> > > > > > > > > > > >
> > > > > > > > > > > > On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
> > > > > > > > > > > > [hidden email] <javascript:;>>
> wrote:
> > > > > > > > > > > >
> > > > > > > > > > > >> +1
> > > > > > > > > > > >>
> > > > > > > > > > > >> Thanks & Regards
> > > > > > > > > > > >> --
> > > > > > > > > > > >> Deepak Dixit
> > > > > > > > > > > >> www.hotwaxsystems.com
> > > > > > > > > > > >>
> > > > > > > > > > > >> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
> > > > > > > > > > > >> [hidden email] <javascript:;>>
> > wrote:
> > > > > > > > > > > >>
> > > > > > > > > > > >> > +1
> > > > > > > > > > > >> >
> > > > > > > > > > > >> > Makes perfect sense.
> > > > > > > > > > > >> >
> > > > > > > > > > > >> > --
> > > > > > > > > > > >> > Thanks & Regards,
> > > > > > > > > > > >> > Mridul Pathak
> > > > > > > > > > > >> > Senior Manager
> > > > > > > > > > > >> > HotWax Systems
> > > > > > > > > > > >> > http://www.hotwaxsystems.com
> > > > > > > > > > > >> >
> > > > > > > > > > > >> > > On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
> > > > > > > > > > > >> [hidden email] <javascript:;>>
> > > > > > > > > > > >> > wrote:
> > > > > > > > > > > >> > >
> > > > > > > > > > > >> > > Hello Everyone,
> > > > > > > > > > > >> > >
> > > > > > > > > > > >> > > After refactoring the start component and while
> > > > starting
> > > > > > on
> > > > > > > > the
> > > > > > > > > > base
> > > > > > > > > > > >> > > component I realized that the testing framework
> > for
> > > > > OFBiz
> > > > > > is
> > > > > > > > not
> > > > > > > > > > > good.
> > > > > > > > > > > >> > You
> > > > > > > > > > > >> > > cannot do real test driven development or
> > > > > > red-green-refactor
> > > > > > > > > with
> > > > > > > > > > > the
> > > > > > > > > > > >> > > current setup, hence my proposal to change it. I
> > > > explain
> > > > > > > > below:
> > > > > > > > > > > >> > >
> > > > > > > > > > > >> > > Problem with current design
> > > > > > > > > > > >> > > ----------------------------------------
> > > > > > > > > > > >> > > - What we have right now is not unit tests, it's
> > > > really
> > > > > > > > > > integration
> > > > > > > > > > > >> > tests.
> > > > > > > > > > > >> > > You have to start the framework, the database,
> the
> > > > > service
> > > > > > > > > engine,
> > > > > > > > > > > the
> > > > > > > > > > > >> > > entity engine and pretty much everything.
> > > > > > > > > > > >> > > - Testing is very slow, because it's an
> > integration
> > > > test
> > > > > > as
> > > > > > > I
> > > > > > > > > > > >> mentioned
> > > > > > > > > > > >> > > above. 10 minutes on a good computer!
> > > > > > > > > > > >> > > - There is zero mocking! We actually have to
> > > > --load-data
> > > > > > for
> > > > > > > > > > things
> > > > > > > > > > > to
> > > > > > > > > > > >> > > work. Again, these are integration tests.
> > > > > > > > > > > >> > > - Too complex: Integration tests by their nature
> > are
> > > > > > > grabbing
> > > > > > > > > too
> > > > > > > > > > > >> much.
> > > > > > > > > > > >> > > Mind you, I am not objecting to integration
> tests
> > (I
> > > > > > > actually
> > > > > > > > > like
> > > > > > > > > > > >> them)
> > > > > > > > > > > >> > > but I am objecting to not having real
> unit-tests.
> > > Unit
> > > > > > tests
> > > > > > > > > > should
> > > > > > > > > > > >> all
> > > > > > > > > > > >> > run
> > > > > > > > > > > >> > > in a few seconds.
> > > > > > > > > > > >> > >
> > > > > > > > > > > >> > > Proposed solution
> > > > > > > > > > > >> > > --------------------------
> > > > > > > > > > > >> > > - We keep what is considered real integration
> > tests
> > > > the
> > > > > > way
> > > > > > > > they
> > > > > > > > > > are
> > > > > > > > > > > >> > right
> > > > > > > > > > > >> > > now and keep using them
> > > > > > > > > > > >> > > - We move what should be unit tests into simple
> > > JUnit
> > > > > > > classes,
> > > > > > > > > and
> > > > > > > > > > > we
> > > > > > > > > > > >> do
> > > > > > > > > > > >> > > not run them using java -jar ofbiz.jar --test,
> but
> > > > > instead
> > > > > > > run
> > > > > > > > > > them
> > > > > > > > > > > >> > > directly from the build.xml script, so these
> files
> > > are
> > > > > not
> > > > > > > > > > > identified
> > > > > > > > > > > >> in
> > > > > > > > > > > >> > > any XML document, but are simply called
> > immediately
> > > > from
> > > > > > the
> > > > > > > > > build
> > > > > > > > > > > >> > scripts.
> > > > > > > > > > > >> > > - We clearly mark the difference between
> > integration
> > > > > tests
> > > > > > > and
> > > > > > > > > > unit
> > > > > > > > > > > >> tests
> > > > > > > > > > > >> > > (inside the source files or in the suite
> > > > declarations).
> > > > > > > > > > > >> > > - We change the run-tests target in build.xml to
> > run
> > > > > both
> > > > > > > unit
> > > > > > > > > > tests
> > > > > > > > > > > >> and
> > > > > > > > > > > >> > > integration tests.
> > > > > > > > > > > >> > >
> > > > > > > > > > > >> > > I intend to heavily refactor the framework and I
> > > would
> > > > > > feel
> > > > > > > > > better
> > > > > > > > > > > >> about
> > > > > > > > > > > >> > > introducing this change while refactoring. What
> do
> > > you
> > > > > > guys
> > > > > > > > > think?
> > > > > > > > > > > >> Ideas?
> > > > > > > > > > > >> > > Suggestions? Approvals and thumbs up?
> > > > > > > > > > > >> > >
> > > > > > > > > > > >> > > Regards,
> > > > > > > > > > > >> > >
> > > > > > > > > > > >> > > Taher Alkhateeb
> > > > > > > > > > > >> >
> > > > > > > > > > > >> >
> > > > > > > > > > > >>
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

[Montalbano Florian]

Hi everyone,

unit tests and integration tests are complementary and with test, the
more the better.
Implementing unit tests will be a great load of work but it is the same
with every kind of test. As Taher said, we will gain a lot from those
unit tests for avoiding regression while refactoring the framework. From
the start component scope, it has already be proved useful.
And what Ron said about encouraging the creation of "smaller class that
can be tested easily" is very important too. It can be a first step to
reach TDD in the OFBiz development. TDD can help to give confidence to
newcomer and veteran when updating some part of the code and as Hans
said, this lead to "continuous improvement and finally continuous
deployment".

I'm aware that this does not solve the technical implementation stuff,
but I think this is an improvement that need to be supported.

Have a nice day,

Florian Montalbano

Le 19/07/2016 09:04, Pierre Smits a écrit :

> As with anything, the law of diminishing returns also applies to OFBiz and
> tests. This is not true for unit tests and system integration tests, but
> also for user acceptance and performance tests.
>
> Nevertheless, the work done up to now is a good start and - I feel
> confident - appreciated. And unit tests are certainly valuable in the
> framework stack. How it will be for functions (regarding components in
> application and special purpose stack) needs to be addressed when we reach
> that bridge.
>
>
> Best regards,
>
> Pierre Smits
>
> ORRTIZ.COM <http://www.orrtiz.com>
> OFBiz based solutions & services
>
> OFBiz Extensions Marketplace
> http://oem.ofbizci.net/oci-2/
>
> On Tue, Jul 19, 2016 at 8:22 AM, Taher Alkhateeb <[hidden email]
>> wrote:
>> Hi Scott,
>>
>> Well spoken thank you. Okay may I suggest that for any such work we will
>> discuss it here see its Merit and if it makes sense then we take it in. It
>> is a little early to discuss it right now because we did not yet go to the
>> higher-level components. Once we do I'll be sure to have a conversation
>> about this in here and would appreciate your input to it.
>>
>> Regards,
>>
>> Taher Alkhateeb
>>
>> On Jul 19, 2016 9:00 AM, "Scott Gray" <[hidden email]>
>> wrote:
>>
>>> Yeah I'm sure unit tests probably worked well for the start component
>> given
>>> it is the lowest level component in the system and closest to a basic
>> java
>>> app.  I just think the value proposition might decrease the further up
>> the
>>> stack we move with them.  I'm not against unit tests when they prove
>>> useful, but further up the stack I think we should prove the case for
>> them
>>> before doing much work to support the mocking that will be required to
>> keep
>>> them inline with 'best practices'.
>>>
>>> In OFBiz "bad or unwanted" tends to come mostly in the way of increasing
>>> complexity from features that require more effort to maintain than the
>>> value they provide to the community.  I think there's a chance attempting
>>> to mock service tests could fall into that bucket.  I could be wrong,
>> but I
>>> think it's worth looking into before we declare that unit tests are the
>>> best form of testing for OFBiz.
>>>
>>> Regards
>>> Scott
>>>
>>> On 19 July 2016 at 17:37, Taher Alkhateeb <[hidden email]>
>>> wrote:
>>>
>>>> Hi Scott,
>>>>
>>>> Thank you for the feedback. To be focused exactly on integration vs
>>> unit, I
>>>> already mentioned above that at least for me the main objective is to
>>>> confidently and quickly run the tests in short bursts of red-green
>>>> refactor. This allows me to refactor code without waiting in front of
>> the
>>>> screen in between test cycles thus giving me immediate feedback on any
>>>> errors I made. Perhaps my intro was too long so this is the squeezed
>> out
>>>> version of it.
>>>>
>>>> I already had one round of testing in the start component which was
>> much
>>>> faster that way and had an immediate impact. Oh and by the way, you
>>> cannot
>>>> test the start component with integration tests for example unless you
>> do
>>>> it from an external component which cannot access package protected
>>> items.
>>>> This style of coding is applicable I think to any software project
>>>> inclusive of OFBiz. Maybe in certain components more than others or
>>> certain
>>>> areas more than others. But I can't see how it could be bad or
>> unwanted.
>>>> Taher Alkhateeb
>>>>
>>>> On Jul 19, 2016 8:18 AM, "Scott Gray" <[hidden email]>
>>>> wrote:
>>>>
>>>>> Thanks Ron and Taher for your responses, I appreciate them but I
>> don't
>>>> hear
>>>>> much in the conversation that speaks to the value of unit tests over
>>>>> integration tests.  Most of the thoughts shared speaks more to the
>>> value
>>>> of
>>>>> tests rather than differences in the type of tests.
>>>>>
>>>>> Speed: At an application level we have ~685 tests that run in 35
>>> seconds
>>>>> (excluding build and data load).  Another point is that there isn't
>>> much
>>>>> reason why tests can't be run in parallel rather than sequentially.
>>>>> TDD: Integration tests in OFBiz are as simple if not simpler to write
>>>> than
>>>>> unit tests and just as useful for TDD.
>>>>> I'm not sure if I missed any other points raised regarding
>> integration
>>>> vs.
>>>>> unit tests?
>>>>>
>>>>> I'm not looking to start a big long debate on the topic, I just
>> wanted
>>> to
>>>>> speak out that someone out there (me) doesn't think unit tests are
>> the
>>>> best
>>>>> solution for testing OFBiz applications.
>>>>>
>>>>> Regards
>>>>> Scott
>>>>>
>>>>> On 19 July 2016 at 16:52, Taher Alkhateeb <
>> [hidden email]>
>>>>> wrote:
>>>>>
>>>>>> Hi Scott,
>>>>>>
>>>>>> Thank you for your input. Your ideas are thought provoking and
>>>> enriching
>>>>> to
>>>>>> the conversation.
>>>>>>
>>>>>> The way I look at it, the general rule is usually many unit tests,
>>> less
>>>>>> integration tests, lesser functional tests. So we are not excluding
>>> any
>>>>>> types of test, all of them are important in certain areas for
>> certain
>>>>>> purposes with certain quantity. Usually integration tests are less
>>>>> because
>>>>>> as you said they just grab more. I like the picture below as a
>>> general
>>>>>> guideline
>>>>>>
>>>>>> http://i.stack.imgur.com/fjQvQ.png
>>>>>>
>>>>>> As you already mentioned unit tests are useful for the framework. I
>>>>>> discovered errors in code that I wrote which I was very very
>> careful
>>>>> with.
>>>>>> I immediately learned the lesson that humans are not designed to
>>> code,
>>>>> and
>>>>>> TDD gives you confidence as you build your code with those short
>>> 30-60
>>>>>> second red-green refactors. I feel much much safer and more
>> confident
>>>>>> writing code that way, the test also documents how to use the api,
>>>>>> refactoring and feature change becomes less terrifying. Also messy
>>> code
>>>>> is
>>>>>> usually not test friendly thus refactoring and unit tests go
>>>> hand-in-hand
>>>>>> for improving the code base.
>>>>>>
>>>>>> Also, I am sorry to say that the framework code is rather messy and
>>>>> brittle
>>>>>> in many places. I think you probably encountered this yourself. The
>>>> same
>>>>> is
>>>>>> said for the applications. Now If we start refactoring without unit
>>>> tests
>>>>>> then we are back to scary business. So much can go wrong so fast
>> and
>>>>> break
>>>>>> things you never expected to break. The framework with all
>>> applications
>>>>>> require heavy refactoring of things like massive ugly methods,
>>>> sandwiched
>>>>>> logic, heavy shared mutable state, hidden dependencies, poor
>>>>>> interfaces, and much more. Every time I touch something I get
>> shocked
>>>> at
>>>>>> how bad it looks, spaghetti logic to no end. If you refactor with
>>>>>> integration tests instead of unit tests then you will come down to
>> a
>>>>>> screeching halt as you wait between these test cycles. My computer
>>>>> actually
>>>>>> takes 10 minutes or more for a full clean load data and testing.
>>>>>>
>>>>>> Now talking about mocking you raise some interesting points, what
>> can
>>>> you
>>>>>> mock vs utilizing integration tests? This is an excellent question
>>> that
>>>>>> does not have a simple answer, and you mostly learn as you code,
>>>>> difficult
>>>>>> to envision without coding. However simple things like Java
>> services
>>>> can
>>>>> be
>>>>>> mocked with a standard mocking class that you use everywhere.
>>>>> Transactions
>>>>>> are difficult to mock and better left to integration tests I think.
>>>> SECAs
>>>>>> might be mocked by simply passing output to input as chaining
>>> methods.
>>>>> Mind
>>>>>> you I am not 100% sure of all of this, coding is the ultimate guide
>>> to
>>>>> what
>>>>>> can or cannot be done.
>>>>>>
>>>>>> So I am not suggesting unit tests as a best-practice (even though
>> it
>>>> is).
>>>>>> Instead I suggest it as something that I and others did and got a
>>> huge
>>>>>> psychological relief and confidence and comfort from. Swimming in
>>> code
>>>>>> without tests is a terrifying business, made more terrifying if the
>>>> code
>>>>> is
>>>>>> bad. Short bursts of red/green/red/green makes you feel good as you
>>>> build
>>>>>> up your logic. And I don't have an exact vision of how to do it,
>>>> because
>>>>>> the details always win.
>>>>>>
>>>>>> I look forward to hearing your thoughts and thank you for enriching
>>>> this
>>>>>> conversation.
>>>>>>
>>>>>> Taher Alkhateeb
>>>>>>
>>>>>> On Tuesday, 19 July 2016, Scott Gray <[hidden email]
>>>>> wrote:
>>>>>>> I know I'm late to the party here, but I just want to say that I
>>>> think
>>>>>>> integration tests have far greater value to OFBiz than unit
>> tests.
>>>>>> Mostly
>>>>>>> because we tend to have quite a low number of tests and
>> integration
>>>>> tests
>>>>>>> give us much better coverage per line of test code and the tests
>>> are
>>>>> much
>>>>>>> closer to the real world scenarios the application might
>> encounter.
>>>>>>> I don't really see how unit tests could be applied to
>> non-framework
>>>>>> testing
>>>>>>> in a useful manner, could you expand on your vision in that
>> regard?
>>>> I
>>>>>> mean
>>>>>>> would we be testing something smaller than a service as the
>> 'unit'?
>>>>> What
>>>>>>> would we mock? Would transaction management still be active? What
>>>>> happens
>>>>>>> when the service calls another service, I guess we mock the
>>> response
>>>>> from
>>>>>>> that service (how)?
>>>>>>>
>>>>>>> It just seems a very complicated method to achieve a less
>> thorough
>>>> but
>>>>>>> albeit faster (maybe) test result.
>>>>>>>
>>>>>>> A build, data load and full test run takes 4m 9s on my laptop.
>>>>> Excluding
>>>>>>> build, data load and framework tests: the application level tests
>>>> take
>>>>>> 35s,
>>>>>>> not very expensive IMO.  The data load time can be reduced to
>>>>> practically
>>>>>>> nothing by copying a clean slate database into runtime for each
>>> run.
>>>>>>> I'm mostly just suggesting we be wary of adding complicated
>> testing
>>>>>>> procedures in the hope of achieving some 'best practice' result
>>> which
>>>>> in
>>>>>>> reality will provide minimal benefits.
>>>>>>>
>>>>>>> Regards
>>>>>>> Scott
>>>>>>>
>>>>>>> On 18 July 2016 at 18:57, Taher Alkhateeb <
>>>> [hidden email]
>>>>>>> <javascript:;>>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hello Akash,
>>>>>>>>
>>>>>>>> Fantastic, I have a few unit tests almost done to be included
>> in
>>>> the
>>>>>>> start
>>>>>>>> component. I will create a new subtask under OFBIZ-1463 to
>> commit
>>>> the
>>>>>>> tests
>>>>>>>> so you can use them as a reference if you like to.
>>>>>>>>
>>>>>>>> I also recommend that you follow the same directory structure
>>>> between
>>>>>> the
>>>>>>>> test code and production code. So for example:
>>>>>>>>
>>>>>>>> Production code:
>>>>>>> framework/start/src/main/java/org/apache/ofbiz/base/start
>>>>>>>> Test code:
>>>> framework/start/src/test/java/org/apache/ofbiz/base/start
>>>>>>>> The benefit of this hierarchy is that you can access non-public
>>>>>> (package
>>>>>>>> protected) methods for testing. This is in fact exactly what I
>>>> needed
>>>>>> to
>>>>>>> be
>>>>>>>> able to apply some of the tests.
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>>
>>>>>>>> Taher Alkhateeb
>>>>>>>>
>>>>>>>> On Mon, Jul 18, 2016 at 9:22 AM, Akash Jain <
>>>>>>> [hidden email] <javascript:;>>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Thanks Taher for nice initiative!
>>>>>>>>>
>>>>>>>>> We are planning to written unit tests to all components under
>>>>>>> OFBIZ-1463
>>>>>>>>> Thanks and Regards
>>>>>>>>> --
>>>>>>>>> Akash Jain
>>>>>>>>>
>>>>>>>>> On Mon, Jul 18, 2016 at 10:36 AM, Taher Alkhateeb <
>>>>>>>>> [hidden email] <javascript:;>> wrote:
>>>>>>>>>
>>>>>>>>>> Hello Everyone,
>>>>>>>>>>
>>>>>>>>>> In reference to this thread and the Jira OFBIZ-7254, I'm
>> very
>>>>> happy
>>>>>>> to
>>>>>>>>>> announce that OFBiz is now ready for applying unit tests
>> with
>>>>> only
>>>>>> 8
>>>>>>>> new
>>>>>>>>>> lines of code in the build script (r1753143) :)
>>>>>>>>>>
>>>>>>>>>> I recommend we do the following moving forward:
>>>>>>>>>>
>>>>>>>>>> 1- Introduce unit tests as much as we can to all components
>>>>>>>>>> 2- Migrate most of the integration tests we currently have
>> to
>>>>> unit
>>>>>>>> tests
>>>>>>>>>> (since they are designed to do very little integration).
>>>>>>>>>>
>>>>>>>>>> This is a great chance for us to practice real TDD in OFBiz
>>>>>>>>>>
>>>>>>>>>> Cheers,
>>>>>>>>>>
>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>
>>>>>>>>>> On Mon, Jun 13, 2016 at 7:43 AM, Pranay Pandey <
>>>>>>>>>> [hidden email] <javascript:;>> wrote:
>>>>>>>>>>
>>>>>>>>>>> +1
>>>>>>>>>>>
>>>>>>>>>>> Best regards,
>>>>>>>>>>>
>>>>>>>>>>> Pranay Pandey
>>>>>>>>>>> HotWax Systems
>>>>>>>>>>> http://www.hotwaxsystems.com/
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Jun 10, 2016 at 7:46 PM, Taher Alkhateeb <
>>>>>>>>>>> [hidden email] <javascript:;>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>> Hello Everyone,
>>>>>>>>>>>>
>>>>>>>>>>>> I was able to get a few tests running and this is very
>>>>> doable.
>>>>>>> But
>>>>>>>> I
>>>>>>>>>>> faced
>>>>>>>>>>>> a big problem in designing the testing framework
>> because
>>> of
>>>>>> ANT.
>>>>>>>>>>>> The problem
>>>>>>>>>>>> ----------------
>>>>>>>>>>>> The way the build scripts are designed in OFBiz are
>> very
>>>>>>> complex. A
>>>>>>>>>>> master
>>>>>>>>>>>> file calls other files which call other files. And in
>> the
>>>>>> middle
>>>>>>>> you
>>>>>>>>>> have
>>>>>>>>>>>> external libraries (ant-contrib) and macros, and
>>> variables,
>>>>> and
>>>>>>>> class
>>>>>>>>>>> path
>>>>>>>>>>>> declarations, and and and ....
>>>>>>>>>>>>
>>>>>>>>>>>> I cannot declare the tests programmatically (with JUnit
>>>> test
>>>>>>>> suites)
>>>>>>>>>>>> because this means lower level components would depend
>> on
>>>>>> higher
>>>>>>>>> level
>>>>>>>>>>>> components. So I have to do it in ANT, by navigating
>> this
>>>>> maze
>>>>>> of
>>>>>>>>> build
>>>>>>>>>>>> scripts, and it was a headache for me just to read
>> them,
>>>> let
>>>>>>> alone
>>>>>>>>>> modify
>>>>>>>>>>>> them to create a testing framework.
>>>>>>>>>>>>
>>>>>>>>>>>> Suggested Solution
>>>>>>>>>>>> ------------------------
>>>>>>>>>>>> I suggest to implement the testing framework in Gradle,
>>> and
>>>>>>> simply
>>>>>>>>> call
>>>>>>>>>>> it
>>>>>>>>>>>> from within ant. This is a middle solution that
>> sustains
>>>> ant
>>>>>> for
>>>>>>>> now,
>>>>>>>>>> but
>>>>>>>>>>>> can allow us to switch out later.
>>>>>>>>>>>>
>>>>>>>>>>>> This means I will just add one more file called
>>>> build.gradle
>>>>> in
>>>>>>> the
>>>>>>>>> top
>>>>>>>>>>>> level directory, and figure out the business logic for
>>>>> calling
>>>>>>> the
>>>>>>>>> test
>>>>>>>>>>>> suites from that file
>>>>>>>>>>>>
>>>>>>>>>>>> I look forward to your feedback.
>>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>>
>>>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <
>>>>>>>>>>>> [hidden email] <javascript:;>>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Everyone,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thank you all for your support, JIRA created in
>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OFBIZ-7254
>>>>>>>>>>>>>
>>>>>>>>>>>>> I will start working on it and try to implement ASAP
>> to
>>>> get
>>>>>> my
>>>>>>>>> focus
>>>>>>>>>>> back
>>>>>>>>>>>>> on refactoring.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Cheers!
>>>>>>>>>>>>>
>>>>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
>>>>>>>>>>>>> [hidden email] <javascript:;>>
>> wrote:
>>>>>>>>>>>>>> +1
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks & Regards
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Deepak Dixit
>>>>>>>>>>>>>> www.hotwaxsystems.com
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
>>>>>>>>>>>>>> [hidden email] <javascript:;>>
>>> wrote:
>>>>>>>>>>>>>>> +1
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Makes perfect sense.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Thanks & Regards,
>>>>>>>>>>>>>>> Mridul Pathak
>>>>>>>>>>>>>>> Senior Manager
>>>>>>>>>>>>>>> HotWax Systems
>>>>>>>>>>>>>>> http://www.hotwaxsystems.com
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
>>>>>>>>>>>>>> [hidden email] <javascript:;>>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> Hello Everyone,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> After refactoring the start component and while
>>>>> starting
>>>>>>> on
>>>>>>>>> the
>>>>>>>>>>> base
>>>>>>>>>>>>>>>> component I realized that the testing framework
>>> for
>>>>>> OFBiz
>>>>>>> is
>>>>>>>>> not
>>>>>>>>>>>> good.
>>>>>>>>>>>>>>> You
>>>>>>>>>>>>>>>> cannot do real test driven development or
>>>>>>> red-green-refactor
>>>>>>>>>> with
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> current setup, hence my proposal to change it. I
>>>>> explain
>>>>>>>>> below:
>>>>>>>>>>>>>>>> Problem with current design
>>>>>>>>>>>>>>>> ----------------------------------------
>>>>>>>>>>>>>>>> - What we have right now is not unit tests, it's
>>>>> really
>>>>>>>>>>> integration
>>>>>>>>>>>>>>> tests.
>>>>>>>>>>>>>>>> You have to start the framework, the database,
>> the
>>>>>> service
>>>>>>>>>> engine,
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> entity engine and pretty much everything.
>>>>>>>>>>>>>>>> - Testing is very slow, because it's an
>>> integration
>>>>> test
>>>>>>> as
>>>>>>>> I
>>>>>>>>>>>>>> mentioned
>>>>>>>>>>>>>>>> above. 10 minutes on a good computer!
>>>>>>>>>>>>>>>> - There is zero mocking! We actually have to
>>>>> --load-data
>>>>>>> for
>>>>>>>>>>> things
>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> work. Again, these are integration tests.
>>>>>>>>>>>>>>>> - Too complex: Integration tests by their nature
>>> are
>>>>>>>> grabbing
>>>>>>>>>> too
>>>>>>>>>>>>>> much.
>>>>>>>>>>>>>>>> Mind you, I am not objecting to integration
>> tests
>>> (I
>>>>>>>> actually
>>>>>>>>>> like
>>>>>>>>>>>>>> them)
>>>>>>>>>>>>>>>> but I am objecting to not having real
>> unit-tests.
>>>> Unit
>>>>>>> tests
>>>>>>>>>>> should
>>>>>>>>>>>>>> all
>>>>>>>>>>>>>>> run
>>>>>>>>>>>>>>>> in a few seconds.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Proposed solution
>>>>>>>>>>>>>>>> --------------------------
>>>>>>>>>>>>>>>> - We keep what is considered real integration
>>> tests
>>>>> the
>>>>>>> way
>>>>>>>>> they
>>>>>>>>>>> are
>>>>>>>>>>>>>>> right
>>>>>>>>>>>>>>>> now and keep using them
>>>>>>>>>>>>>>>> - We move what should be unit tests into simple
>>>> JUnit
>>>>>>>> classes,
>>>>>>>>>> and
>>>>>>>>>>>> we
>>>>>>>>>>>>>> do
>>>>>>>>>>>>>>>> not run them using java -jar ofbiz.jar --test,
>> but
>>>>>> instead
>>>>>>>> run
>>>>>>>>>>> them
>>>>>>>>>>>>>>>> directly from the build.xml script, so these
>> files
>>>> are
>>>>>> not
>>>>>>>>>>>> identified
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>> any XML document, but are simply called
>>> immediately
>>>>> from
>>>>>>> the
>>>>>>>>>> build
>>>>>>>>>>>>>>> scripts.
>>>>>>>>>>>>>>>> - We clearly mark the difference between
>>> integration
>>>>>> tests
>>>>>>>> and
>>>>>>>>>>> unit
>>>>>>>>>>>>>> tests
>>>>>>>>>>>>>>>> (inside the source files or in the suite
>>>>> declarations).
>>>>>>>>>>>>>>>> - We change the run-tests target in build.xml to
>>> run
>>>>>> both
>>>>>>>> unit
>>>>>>>>>>> tests
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>> integration tests.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I intend to heavily refactor the framework and I
>>>> would
>>>>>>> feel
>>>>>>>>>> better
>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>> introducing this change while refactoring. What
>> do
>>>> you
>>>>>>> guys
>>>>>>>>>> think?
>>>>>>>>>>>>>> Ideas?
>>>>>>>>>>>>>>>> Suggestions? Approvals and thumbs up?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>

[Scott Gray-3]

I think people are missing my point because they keep replying with generic
statements that aren't specific to unit tests.

On 19 July 2016 at 20:08, Montalbano Florian <[hidden email]>
wrote:

> Hi everyone,
>
> unit tests and integration tests are complementary and with test, the more
> the better.
> Implementing unit tests will be a great load of work but it is the same
> with every kind of test. As Taher said, we will gain a lot from those unit
> tests for avoiding regression while refactoring the framework. From the
> start component scope, it has already be proved useful.
> And what Ron said about encouraging the creation of "smaller class that
> can be tested easily" is very important too. It can be a first step to
> reach TDD in the OFBiz development. TDD can help to give confidence to
> newcomer and veteran when updating some part of the code and as Hans said,
> this lead to "continuous improvement and finally continuous deployment".
>
> I'm aware that this does not solve the technical implementation stuff, but
> I think this is an improvement that need to be supported.
>
> Have a nice day,
>
> Florian Montalbano
>
>
> Le 19/07/2016 09:04, Pierre Smits a écrit :
>
>> As with anything, the law of diminishing returns also applies to OFBiz and
>> tests. This is not true for unit tests and system integration tests, but
>> also for user acceptance and performance tests.
>>
>> Nevertheless, the work done up to now is a good start and - I feel
>> confident - appreciated. And unit tests are certainly valuable in the
>> framework stack. How it will be for functions (regarding components in
>> application and special purpose stack) needs to be addressed when we reach
>> that bridge.
>>
>>
>> Best regards,
>>
>> Pierre Smits
>>
>> ORRTIZ.COM <http://www.orrtiz.com>
>> OFBiz based solutions & services
>>
>> OFBiz Extensions Marketplace
>> http://oem.ofbizci.net/oci-2/
>>
>> On Tue, Jul 19, 2016 at 8:22 AM, Taher Alkhateeb <
>> [hidden email]
>>
>>> wrote:
>>> Hi Scott,
>>>
>>> Well spoken thank you. Okay may I suggest that for any such work we will
>>> discuss it here see its Merit and if it makes sense then we take it in.
>>> It
>>> is a little early to discuss it right now because we did not yet go to
>>> the
>>> higher-level components. Once we do I'll be sure to have a conversation
>>> about this in here and would appreciate your input to it.
>>>
>>> Regards,
>>>
>>> Taher Alkhateeb
>>>
>>> On Jul 19, 2016 9:00 AM, "Scott Gray" <[hidden email]>
>>> wrote:
>>>
>>> Yeah I'm sure unit tests probably worked well for the start component
>>>>
>>> given
>>>
>>>> it is the lowest level component in the system and closest to a basic
>>>>
>>> java
>>>
>>>> app.  I just think the value proposition might decrease the further up
>>>>
>>> the
>>>
>>>> stack we move with them.  I'm not against unit tests when they prove
>>>> useful, but further up the stack I think we should prove the case for
>>>>
>>> them
>>>
>>>> before doing much work to support the mocking that will be required to
>>>>
>>> keep
>>>
>>>> them inline with 'best practices'.
>>>>
>>>> In OFBiz "bad or unwanted" tends to come mostly in the way of increasing
>>>> complexity from features that require more effort to maintain than the
>>>> value they provide to the community.  I think there's a chance
>>>> attempting
>>>> to mock service tests could fall into that bucket.  I could be wrong,
>>>>
>>> but I
>>>
>>>> think it's worth looking into before we declare that unit tests are the
>>>> best form of testing for OFBiz.
>>>>
>>>> Regards
>>>> Scott
>>>>
>>>> On 19 July 2016 at 17:37, Taher Alkhateeb <[hidden email]>
>>>> wrote:
>>>>
>>>> Hi Scott,
>>>>>
>>>>> Thank you for the feedback. To be focused exactly on integration vs
>>>>>
>>>> unit, I
>>>>
>>>>> already mentioned above that at least for me the main objective is to
>>>>> confidently and quickly run the tests in short bursts of red-green
>>>>> refactor. This allows me to refactor code without waiting in front of
>>>>>
>>>> the
>>>
>>>> screen in between test cycles thus giving me immediate feedback on any
>>>>> errors I made. Perhaps my intro was too long so this is the squeezed
>>>>>
>>>> out
>>>
>>>> version of it.
>>>>>
>>>>> I already had one round of testing in the start component which was
>>>>>
>>>> much
>>>
>>>> faster that way and had an immediate impact. Oh and by the way, you
>>>>>
>>>> cannot
>>>>
>>>>> test the start component with integration tests for example unless you
>>>>>
>>>> do
>>>
>>>> it from an external component which cannot access package protected
>>>>>
>>>> items.
>>>>
>>>>> This style of coding is applicable I think to any software project
>>>>> inclusive of OFBiz. Maybe in certain components more than others or
>>>>>
>>>> certain
>>>>
>>>>> areas more than others. But I can't see how it could be bad or
>>>>>
>>>> unwanted.
>>>
>>>> Taher Alkhateeb
>>>>>
>>>>> On Jul 19, 2016 8:18 AM, "Scott Gray" <[hidden email]>
>>>>> wrote:
>>>>>
>>>>> Thanks Ron and Taher for your responses, I appreciate them but I
>>>>>>
>>>>> don't
>>>
>>>> hear
>>>>>
>>>>>> much in the conversation that speaks to the value of unit tests over
>>>>>> integration tests.  Most of the thoughts shared speaks more to the
>>>>>>
>>>>> value
>>>>
>>>>> of
>>>>>
>>>>>> tests rather than differences in the type of tests.
>>>>>>
>>>>>> Speed: At an application level we have ~685 tests that run in 35
>>>>>>
>>>>> seconds
>>>>
>>>>> (excluding build and data load).  Another point is that there isn't
>>>>>>
>>>>> much
>>>>
>>>>> reason why tests can't be run in parallel rather than sequentially.
>>>>>> TDD: Integration tests in OFBiz are as simple if not simpler to write
>>>>>>
>>>>> than
>>>>>
>>>>>> unit tests and just as useful for TDD.
>>>>>> I'm not sure if I missed any other points raised regarding
>>>>>>
>>>>> integration
>>>
>>>> vs.
>>>>>
>>>>>> unit tests?
>>>>>>
>>>>>> I'm not looking to start a big long debate on the topic, I just
>>>>>>
>>>>> wanted
>>>
>>>> to
>>>>
>>>>> speak out that someone out there (me) doesn't think unit tests are
>>>>>>
>>>>> the
>>>
>>>> best
>>>>>
>>>>>> solution for testing OFBiz applications.
>>>>>>
>>>>>> Regards
>>>>>> Scott
>>>>>>
>>>>>> On 19 July 2016 at 16:52, Taher Alkhateeb <
>>>>>>
>>>>> [hidden email]>
>>>
>>>> wrote:
>>>>>>
>>>>>> Hi Scott,
>>>>>>>
>>>>>>> Thank you for your input. Your ideas are thought provoking and
>>>>>>>
>>>>>> enriching
>>>>>
>>>>>> to
>>>>>>
>>>>>>> the conversation.
>>>>>>>
>>>>>>> The way I look at it, the general rule is usually many unit tests,
>>>>>>>
>>>>>> less
>>>>
>>>>> integration tests, lesser functional tests. So we are not excluding
>>>>>>>
>>>>>> any
>>>>
>>>>> types of test, all of them are important in certain areas for
>>>>>>>
>>>>>> certain
>>>
>>>> purposes with certain quantity. Usually integration tests are less
>>>>>>>
>>>>>> because
>>>>>>
>>>>>>> as you said they just grab more. I like the picture below as a
>>>>>>>
>>>>>> general
>>>>
>>>>> guideline
>>>>>>>
>>>>>>> http://i.stack.imgur.com/fjQvQ.png
>>>>>>>
>>>>>>> As you already mentioned unit tests are useful for the framework. I
>>>>>>> discovered errors in code that I wrote which I was very very
>>>>>>>
>>>>>> careful
>>>
>>>> with.
>>>>>>
>>>>>>> I immediately learned the lesson that humans are not designed to
>>>>>>>
>>>>>> code,
>>>>
>>>>> and
>>>>>>
>>>>>>> TDD gives you confidence as you build your code with those short
>>>>>>>
>>>>>> 30-60
>>>>
>>>>> second red-green refactors. I feel much much safer and more
>>>>>>>
>>>>>> confident
>>>
>>>> writing code that way, the test also documents how to use the api,
>>>>>>> refactoring and feature change becomes less terrifying. Also messy
>>>>>>>
>>>>>> code
>>>>
>>>>> is
>>>>>>
>>>>>>> usually not test friendly thus refactoring and unit tests go
>>>>>>>
>>>>>> hand-in-hand
>>>>>
>>>>>> for improving the code base.
>>>>>>>
>>>>>>> Also, I am sorry to say that the framework code is rather messy and
>>>>>>>
>>>>>> brittle
>>>>>>
>>>>>>> in many places. I think you probably encountered this yourself. The
>>>>>>>
>>>>>> same
>>>>>
>>>>>> is
>>>>>>
>>>>>>> said for the applications. Now If we start refactoring without unit
>>>>>>>
>>>>>> tests
>>>>>
>>>>>> then we are back to scary business. So much can go wrong so fast
>>>>>>>
>>>>>> and
>>>
>>>> break
>>>>>>
>>>>>>> things you never expected to break. The framework with all
>>>>>>>
>>>>>> applications
>>>>
>>>>> require heavy refactoring of things like massive ugly methods,
>>>>>>>
>>>>>> sandwiched
>>>>>
>>>>>> logic, heavy shared mutable state, hidden dependencies, poor
>>>>>>> interfaces, and much more. Every time I touch something I get
>>>>>>>
>>>>>> shocked
>>>
>>>> at
>>>>>
>>>>>> how bad it looks, spaghetti logic to no end. If you refactor with
>>>>>>> integration tests instead of unit tests then you will come down to
>>>>>>>
>>>>>> a
>>>
>>>> screeching halt as you wait between these test cycles. My computer
>>>>>>>
>>>>>> actually
>>>>>>
>>>>>>> takes 10 minutes or more for a full clean load data and testing.
>>>>>>>
>>>>>>> Now talking about mocking you raise some interesting points, what
>>>>>>>
>>>>>> can
>>>
>>>> you
>>>>>
>>>>>> mock vs utilizing integration tests? This is an excellent question
>>>>>>>
>>>>>> that
>>>>
>>>>> does not have a simple answer, and you mostly learn as you code,
>>>>>>>
>>>>>> difficult
>>>>>>
>>>>>>> to envision without coding. However simple things like Java
>>>>>>>
>>>>>> services
>>>
>>>> can
>>>>>
>>>>>> be
>>>>>>
>>>>>>> mocked with a standard mocking class that you use everywhere.
>>>>>>>
>>>>>> Transactions
>>>>>>
>>>>>>> are difficult to mock and better left to integration tests I think.
>>>>>>>
>>>>>> SECAs
>>>>>
>>>>>> might be mocked by simply passing output to input as chaining
>>>>>>>
>>>>>> methods.
>>>>
>>>>> Mind
>>>>>>
>>>>>>> you I am not 100% sure of all of this, coding is the ultimate guide
>>>>>>>
>>>>>> to
>>>>
>>>>> what
>>>>>>
>>>>>>> can or cannot be done.
>>>>>>>
>>>>>>> So I am not suggesting unit tests as a best-practice (even though
>>>>>>>
>>>>>> it
>>>
>>>> is).
>>>>>
>>>>>> Instead I suggest it as something that I and others did and got a
>>>>>>>
>>>>>> huge
>>>>
>>>>> psychological relief and confidence and comfort from. Swimming in
>>>>>>>
>>>>>> code
>>>>
>>>>> without tests is a terrifying business, made more terrifying if the
>>>>>>>
>>>>>> code
>>>>>
>>>>>> is
>>>>>>
>>>>>>> bad. Short bursts of red/green/red/green makes you feel good as you
>>>>>>>
>>>>>> build
>>>>>
>>>>>> up your logic. And I don't have an exact vision of how to do it,
>>>>>>>
>>>>>> because
>>>>>
>>>>>> the details always win.
>>>>>>>
>>>>>>> I look forward to hearing your thoughts and thank you for enriching
>>>>>>>
>>>>>> this
>>>>>
>>>>>> conversation.
>>>>>>>
>>>>>>> Taher Alkhateeb
>>>>>>>
>>>>>>> On Tuesday, 19 July 2016, Scott Gray <[hidden email]
>>>>>>>
>>>>>> wrote:
>>>>>>
>>>>>>> I know I'm late to the party here, but I just want to say that I
>>>>>>>>
>>>>>>> think
>>>>>
>>>>>> integration tests have far greater value to OFBiz than unit
>>>>>>>>
>>>>>>> tests.
>>>
>>>> Mostly
>>>>>>>
>>>>>>>> because we tend to have quite a low number of tests and
>>>>>>>>
>>>>>>> integration
>>>
>>>> tests
>>>>>>
>>>>>>> give us much better coverage per line of test code and the tests
>>>>>>>>
>>>>>>> are
>>>>
>>>>> much
>>>>>>
>>>>>>> closer to the real world scenarios the application might
>>>>>>>>
>>>>>>> encounter.
>>>
>>>> I don't really see how unit tests could be applied to
>>>>>>>>
>>>>>>> non-framework
>>>
>>>> testing
>>>>>>>
>>>>>>>> in a useful manner, could you expand on your vision in that
>>>>>>>>
>>>>>>> regard?
>>>
>>>> I
>>>>>
>>>>>> mean
>>>>>>>
>>>>>>>> would we be testing something smaller than a service as the
>>>>>>>>
>>>>>>> 'unit'?
>>>
>>>> What
>>>>>>
>>>>>>> would we mock? Would transaction management still be active? What
>>>>>>>>
>>>>>>> happens
>>>>>>
>>>>>>> when the service calls another service, I guess we mock the
>>>>>>>>
>>>>>>> response
>>>>
>>>>> from
>>>>>>
>>>>>>> that service (how)?
>>>>>>>>
>>>>>>>> It just seems a very complicated method to achieve a less
>>>>>>>>
>>>>>>> thorough
>>>
>>>> but
>>>>>
>>>>>> albeit faster (maybe) test result.
>>>>>>>>
>>>>>>>> A build, data load and full test run takes 4m 9s on my laptop.
>>>>>>>>
>>>>>>> Excluding
>>>>>>
>>>>>>> build, data load and framework tests: the application level tests
>>>>>>>>
>>>>>>> take
>>>>>
>>>>>> 35s,
>>>>>>>
>>>>>>>> not very expensive IMO.  The data load time can be reduced to
>>>>>>>>
>>>>>>> practically
>>>>>>
>>>>>>> nothing by copying a clean slate database into runtime for each
>>>>>>>>
>>>>>>> run.
>>>>
>>>>> I'm mostly just suggesting we be wary of adding complicated
>>>>>>>>
>>>>>>> testing
>>>
>>>> procedures in the hope of achieving some 'best practice' result
>>>>>>>>
>>>>>>> which
>>>>
>>>>> in
>>>>>>
>>>>>>> reality will provide minimal benefits.
>>>>>>>>
>>>>>>>> Regards
>>>>>>>> Scott
>>>>>>>>
>>>>>>>> On 18 July 2016 at 18:57, Taher Alkhateeb <
>>>>>>>>
>>>>>>> [hidden email]
>>>>>
>>>>>> <javascript:;>>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Hello Akash,
>>>>>>>>>
>>>>>>>>> Fantastic, I have a few unit tests almost done to be included
>>>>>>>>>
>>>>>>>> in
>>>
>>>> the
>>>>>
>>>>>> start
>>>>>>>>
>>>>>>>>> component. I will create a new subtask under OFBIZ-1463 to
>>>>>>>>>
>>>>>>>> commit
>>>
>>>> the
>>>>>
>>>>>> tests
>>>>>>>>
>>>>>>>>> so you can use them as a reference if you like to.
>>>>>>>>>
>>>>>>>>> I also recommend that you follow the same directory structure
>>>>>>>>>
>>>>>>>> between
>>>>>
>>>>>> the
>>>>>>>
>>>>>>>> test code and production code. So for example:
>>>>>>>>>
>>>>>>>>> Production code:
>>>>>>>>>
>>>>>>>> framework/start/src/main/java/org/apache/ofbiz/base/start
>>>>>>>>
>>>>>>>>> Test code:
>>>>>>>>>
>>>>>>>> framework/start/src/test/java/org/apache/ofbiz/base/start
>>>>>
>>>>>> The benefit of this hierarchy is that you can access non-public
>>>>>>>>>
>>>>>>>> (package
>>>>>>>
>>>>>>>> protected) methods for testing. This is in fact exactly what I
>>>>>>>>>
>>>>>>>> needed
>>>>>
>>>>>> to
>>>>>>>
>>>>>>>> be
>>>>>>>>
>>>>>>>>> able to apply some of the tests.
>>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>>
>>>>>>>>> Taher Alkhateeb
>>>>>>>>>
>>>>>>>>> On Mon, Jul 18, 2016 at 9:22 AM, Akash Jain <
>>>>>>>>>
>>>>>>>> [hidden email] <javascript:;>>
>>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Thanks Taher for nice initiative!
>>>>>>>>>>
>>>>>>>>>> We are planning to written unit tests to all components under
>>>>>>>>>>
>>>>>>>>> OFBIZ-1463
>>>>>>>>
>>>>>>>>> Thanks and Regards
>>>>>>>>>> --
>>>>>>>>>> Akash Jain
>>>>>>>>>>
>>>>>>>>>> On Mon, Jul 18, 2016 at 10:36 AM, Taher Alkhateeb <
>>>>>>>>>> [hidden email] <javascript:;>> wrote:
>>>>>>>>>>
>>>>>>>>>> Hello Everyone,
>>>>>>>>>>>
>>>>>>>>>>> In reference to this thread and the Jira OFBIZ-7254, I'm
>>>>>>>>>>>
>>>>>>>>>> very
>>>
>>>> happy
>>>>>>
>>>>>>> to
>>>>>>>>
>>>>>>>>> announce that OFBiz is now ready for applying unit tests
>>>>>>>>>>>
>>>>>>>>>> with
>>>
>>>> only
>>>>>>
>>>>>>> 8
>>>>>>>
>>>>>>>> new
>>>>>>>>>
>>>>>>>>>> lines of code in the build script (r1753143) :)
>>>>>>>>>>>
>>>>>>>>>>> I recommend we do the following moving forward:
>>>>>>>>>>>
>>>>>>>>>>> 1- Introduce unit tests as much as we can to all components
>>>>>>>>>>> 2- Migrate most of the integration tests we currently have
>>>>>>>>>>>
>>>>>>>>>> to
>>>
>>>> unit
>>>>>>
>>>>>>> tests
>>>>>>>>>
>>>>>>>>>> (since they are designed to do very little integration).
>>>>>>>>>>>
>>>>>>>>>>> This is a great chance for us to practice real TDD in OFBiz
>>>>>>>>>>>
>>>>>>>>>>> Cheers,
>>>>>>>>>>>
>>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Jun 13, 2016 at 7:43 AM, Pranay Pandey <
>>>>>>>>>>> [hidden email] <javascript:;>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> +1
>>>>>>>>>>>>
>>>>>>>>>>>> Best regards,
>>>>>>>>>>>>
>>>>>>>>>>>> Pranay Pandey
>>>>>>>>>>>> HotWax Systems
>>>>>>>>>>>> http://www.hotwaxsystems.com/
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Jun 10, 2016 at 7:46 PM, Taher Alkhateeb <
>>>>>>>>>>>> [hidden email] <javascript:;>
>>>>>>>>>>>>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> Hello Everyone,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I was able to get a few tests running and this is very
>>>>>>>>>>>>>
>>>>>>>>>>>> doable.
>>>>>>
>>>>>>> But
>>>>>>>>
>>>>>>>>> I
>>>>>>>>>
>>>>>>>>>> faced
>>>>>>>>>>>>
>>>>>>>>>>>>> a big problem in designing the testing framework
>>>>>>>>>>>>>
>>>>>>>>>>>> because
>>>
>>>> of
>>>>
>>>>> ANT.
>>>>>>>
>>>>>>>> The problem
>>>>>>>>>>>>> ----------------
>>>>>>>>>>>>> The way the build scripts are designed in OFBiz are
>>>>>>>>>>>>>
>>>>>>>>>>>> very
>>>
>>>> complex. A
>>>>>>>>
>>>>>>>>> master
>>>>>>>>>>>>
>>>>>>>>>>>>> file calls other files which call other files. And in
>>>>>>>>>>>>>
>>>>>>>>>>>> the
>>>
>>>> middle
>>>>>>>
>>>>>>>> you
>>>>>>>>>
>>>>>>>>>> have
>>>>>>>>>>>
>>>>>>>>>>>> external libraries (ant-contrib) and macros, and
>>>>>>>>>>>>>
>>>>>>>>>>>> variables,
>>>>
>>>>> and
>>>>>>
>>>>>>> class
>>>>>>>>>
>>>>>>>>>> path
>>>>>>>>>>>>
>>>>>>>>>>>>> declarations, and and and ....
>>>>>>>>>>>>>
>>>>>>>>>>>>> I cannot declare the tests programmatically (with JUnit
>>>>>>>>>>>>>
>>>>>>>>>>>> test
>>>>>
>>>>>> suites)
>>>>>>>>>
>>>>>>>>>> because this means lower level components would depend
>>>>>>>>>>>>>
>>>>>>>>>>>> on
>>>
>>>> higher
>>>>>>>
>>>>>>>> level
>>>>>>>>>>
>>>>>>>>>>> components. So I have to do it in ANT, by navigating
>>>>>>>>>>>>>
>>>>>>>>>>>> this
>>>
>>>> maze
>>>>>>
>>>>>>> of
>>>>>>>
>>>>>>>> build
>>>>>>>>>>
>>>>>>>>>>> scripts, and it was a headache for me just to read
>>>>>>>>>>>>>
>>>>>>>>>>>> them,
>>>
>>>> let
>>>>>
>>>>>> alone
>>>>>>>>
>>>>>>>>> modify
>>>>>>>>>>>
>>>>>>>>>>>> them to create a testing framework.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Suggested Solution
>>>>>>>>>>>>> ------------------------
>>>>>>>>>>>>> I suggest to implement the testing framework in Gradle,
>>>>>>>>>>>>>
>>>>>>>>>>>> and
>>>>
>>>>> simply
>>>>>>>>
>>>>>>>>> call
>>>>>>>>>>
>>>>>>>>>>> it
>>>>>>>>>>>>
>>>>>>>>>>>>> from within ant. This is a middle solution that
>>>>>>>>>>>>>
>>>>>>>>>>>> sustains
>>>
>>>> ant
>>>>>
>>>>>> for
>>>>>>>
>>>>>>>> now,
>>>>>>>>>
>>>>>>>>>> but
>>>>>>>>>>>
>>>>>>>>>>>> can allow us to switch out later.
>>>>>>>>>>>>>
>>>>>>>>>>>>> This means I will just add one more file called
>>>>>>>>>>>>>
>>>>>>>>>>>> build.gradle
>>>>>
>>>>>> in
>>>>>>
>>>>>>> the
>>>>>>>>
>>>>>>>>> top
>>>>>>>>>>
>>>>>>>>>>> level directory, and figure out the business logic for
>>>>>>>>>>>>>
>>>>>>>>>>>> calling
>>>>>>
>>>>>>> the
>>>>>>>>
>>>>>>>>> test
>>>>>>>>>>
>>>>>>>>>>> suites from that file
>>>>>>>>>>>>>
>>>>>>>>>>>>> I look forward to your feedback.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <
>>>>>>>>>>>>> [hidden email] <javascript:;>>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Everyone,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thank you all for your support, JIRA created in
>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OFBIZ-7254
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I will start working on it and try to implement ASAP
>>>>>>>>>>>>>>
>>>>>>>>>>>>> to
>>>
>>>> get
>>>>>
>>>>>> my
>>>>>>>
>>>>>>>> focus
>>>>>>>>>>
>>>>>>>>>>> back
>>>>>>>>>>>>
>>>>>>>>>>>>> on refactoring.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Cheers!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
>>>>>>>>>>>>>> [hidden email] <javascript:;>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>> wrote:
>>>
>>>> +1
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks & Regards
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Deepak Dixit
>>>>>>>>>>>>>>> www.hotwaxsystems.com
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
>>>>>>>>>>>>>>> [hidden email] <javascript:;>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> wrote:
>>>>
>>>>> +1
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Makes perfect sense.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> Thanks & Regards,
>>>>>>>>>>>>>>>> Mridul Pathak
>>>>>>>>>>>>>>>> Senior Manager
>>>>>>>>>>>>>>>> HotWax Systems
>>>>>>>>>>>>>>>> http://www.hotwaxsystems.com
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> [hidden email] <javascript:;>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hello Everyone,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> After refactoring the start component and while
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> starting
>>>>>>
>>>>>>> on
>>>>>>>>
>>>>>>>>> the
>>>>>>>>>>
>>>>>>>>>>> base
>>>>>>>>>>>>
>>>>>>>>>>>>> component I realized that the testing framework
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> for
>>>>
>>>>> OFBiz
>>>>>>>
>>>>>>>> is
>>>>>>>>
>>>>>>>>> not
>>>>>>>>>>
>>>>>>>>>>> good.
>>>>>>>>>>>>>
>>>>>>>>>>>>>> You
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> cannot do real test driven development or
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> red-green-refactor
>>>>>>>>
>>>>>>>>> with
>>>>>>>>>>>
>>>>>>>>>>>> the
>>>>>>>>>>>>>
>>>>>>>>>>>>>> current setup, hence my proposal to change it. I
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> explain
>>>>>>
>>>>>>> below:
>>>>>>>>>>
>>>>>>>>>>> Problem with current design
>>>>>>>>>>>>>>>>> ----------------------------------------
>>>>>>>>>>>>>>>>> - What we have right now is not unit tests, it's
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> really
>>>>>>
>>>>>>> integration
>>>>>>>>>>>>
>>>>>>>>>>>>> tests.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> You have to start the framework, the database,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> the
>>>
>>>> service
>>>>>>>
>>>>>>>> engine,
>>>>>>>>>>>
>>>>>>>>>>>> the
>>>>>>>>>>>>>
>>>>>>>>>>>>>> entity engine and pretty much everything.
>>>>>>>>>>>>>>>>> - Testing is very slow, because it's an
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> integration
>>>>
>>>>> test
>>>>>>
>>>>>>> as
>>>>>>>>
>>>>>>>>> I
>>>>>>>>>
>>>>>>>>>> mentioned
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> above. 10 minutes on a good computer!
>>>>>>>>>>>>>>>>> - There is zero mocking! We actually have to
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --load-data
>>>>>>
>>>>>>> for
>>>>>>>>
>>>>>>>>> things
>>>>>>>>>>>>
>>>>>>>>>>>>> to
>>>>>>>>>>>>>
>>>>>>>>>>>>>> work. Again, these are integration tests.
>>>>>>>>>>>>>>>>> - Too complex: Integration tests by their nature
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> are
>>>>
>>>>> grabbing
>>>>>>>>>
>>>>>>>>>> too
>>>>>>>>>>>
>>>>>>>>>>>> much.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Mind you, I am not objecting to integration
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> tests
>>>
>>>> (I
>>>>
>>>>> actually
>>>>>>>>>
>>>>>>>>>> like
>>>>>>>>>>>
>>>>>>>>>>>> them)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> but I am objecting to not having real
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> unit-tests.
>>>
>>>> Unit
>>>>>
>>>>>> tests
>>>>>>>>
>>>>>>>>> should
>>>>>>>>>>>>
>>>>>>>>>>>>> all
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> run
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> in a few seconds.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Proposed solution
>>>>>>>>>>>>>>>>> --------------------------
>>>>>>>>>>>>>>>>> - We keep what is considered real integration
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> tests
>>>>
>>>>> the
>>>>>>
>>>>>>> way
>>>>>>>>
>>>>>>>>> they
>>>>>>>>>>
>>>>>>>>>>> are
>>>>>>>>>>>>
>>>>>>>>>>>>> right
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> now and keep using them
>>>>>>>>>>>>>>>>> - We move what should be unit tests into simple
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> JUnit
>>>>>
>>>>>> classes,
>>>>>>>>>
>>>>>>>>>> and
>>>>>>>>>>>
>>>>>>>>>>>> we
>>>>>>>>>>>>>
>>>>>>>>>>>>>> do
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> not run them using java -jar ofbiz.jar --test,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> but
>>>
>>>> instead
>>>>>>>
>>>>>>>> run
>>>>>>>>>
>>>>>>>>>> them
>>>>>>>>>>>>
>>>>>>>>>>>>> directly from the build.xml script, so these
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> files
>>>
>>>> are
>>>>>
>>>>>> not
>>>>>>>
>>>>>>>> identified
>>>>>>>>>>>>>
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> any XML document, but are simply called
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> immediately
>>>>
>>>>> from
>>>>>>
>>>>>>> the
>>>>>>>>
>>>>>>>>> build
>>>>>>>>>>>
>>>>>>>>>>>> scripts.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> - We clearly mark the difference between
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> integration
>>>>
>>>>> tests
>>>>>>>
>>>>>>>> and
>>>>>>>>>
>>>>>>>>>> unit
>>>>>>>>>>>>
>>>>>>>>>>>>> tests
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> (inside the source files or in the suite
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> declarations).
>>>>>>
>>>>>>> - We change the run-tests target in build.xml to
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> run
>>>>
>>>>> both
>>>>>>>
>>>>>>>> unit
>>>>>>>>>
>>>>>>>>>> tests
>>>>>>>>>>>>
>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> integration tests.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I intend to heavily refactor the framework and I
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> would
>>>>>
>>>>>> feel
>>>>>>>>
>>>>>>>>> better
>>>>>>>>>>>
>>>>>>>>>>>> about
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> introducing this change while refactoring. What
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> do
>>>
>>>> you
>>>>>
>>>>>> guys
>>>>>>>>
>>>>>>>>> think?
>>>>>>>>>>>
>>>>>>>>>>>> Ideas?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Suggestions? Approvals and thumbs up?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>

[Ron Wheeler]

It is not clear what you are expecting as a specific response.

As you move up the food chain, the definition of "units" changes and the
tests get more complicated and the design of the tests gets more complex
(mock objects, testing race conditions).

It is hard to make specific comments about what kind of testing will
give the highest level of confidence at each level but we are going in
the right direction.

The fact that we are considering tests as a major development activity
is a big step forward.

We can argue about specific cases (scope, tools, value, etc) as we get
to them.

I think that we are at the "good idea" stage rather than at the "policy"
stage at this point.

Ron


On 19/07/2016 4:23 AM, Scott Gray wrote:

> I think people are missing my point because they keep replying with generic
> statements that aren't specific to unit tests.
>
> On 19 July 2016 at 20:08, Montalbano Florian <[hidden email]>
> wrote:
>
>> Hi everyone,
>>
>> unit tests and integration tests are complementary and with test, the more
>> the better.
>> Implementing unit tests will be a great load of work but it is the same
>> with every kind of test. As Taher said, we will gain a lot from those unit
>> tests for avoiding regression while refactoring the framework. From the
>> start component scope, it has already be proved useful.
>> And what Ron said about encouraging the creation of "smaller class that
>> can be tested easily" is very important too. It can be a first step to
>> reach TDD in the OFBiz development. TDD can help to give confidence to
>> newcomer and veteran when updating some part of the code and as Hans said,
>> this lead to "continuous improvement and finally continuous deployment".
>>
>> I'm aware that this does not solve the technical implementation stuff, but
>> I think this is an improvement that need to be supported.
>>
>> Have a nice day,
>>
>> Florian Montalbano
>>
>>
>> Le 19/07/2016 09:04, Pierre Smits a écrit :
>>
>>> As with anything, the law of diminishing returns also applies to OFBiz and
>>> tests. This is not true for unit tests and system integration tests, but
>>> also for user acceptance and performance tests.
>>>
>>> Nevertheless, the work done up to now is a good start and - I feel
>>> confident - appreciated. And unit tests are certainly valuable in the
>>> framework stack. How it will be for functions (regarding components in
>>> application and special purpose stack) needs to be addressed when we reach
>>> that bridge.
>>>
>>>
>>> Best regards,
>>>
>>> Pierre Smits
>>>
>>> ORRTIZ.COM <http://www.orrtiz.com>
>>> OFBiz based solutions & services
>>>
>>> OFBiz Extensions Marketplace
>>> http://oem.ofbizci.net/oci-2/
>>>
>>> On Tue, Jul 19, 2016 at 8:22 AM, Taher Alkhateeb <
>>> [hidden email]
>>>
>>>> wrote:
>>>> Hi Scott,
>>>>
>>>> Well spoken thank you. Okay may I suggest that for any such work we will
>>>> discuss it here see its Merit and if it makes sense then we take it in.
>>>> It
>>>> is a little early to discuss it right now because we did not yet go to
>>>> the
>>>> higher-level components. Once we do I'll be sure to have a conversation
>>>> about this in here and would appreciate your input to it.
>>>>
>>>> Regards,
>>>>
>>>> Taher Alkhateeb
>>>>
>>>> On Jul 19, 2016 9:00 AM, "Scott Gray" <[hidden email]>
>>>> wrote:
>>>>
>>>> Yeah I'm sure unit tests probably worked well for the start component
>>>> given
>>>>
>>>>> it is the lowest level component in the system and closest to a basic
>>>>>
>>>> java
>>>>
>>>>> app.  I just think the value proposition might decrease the further up
>>>>>
>>>> the
>>>>
>>>>> stack we move with them.  I'm not against unit tests when they prove
>>>>> useful, but further up the stack I think we should prove the case for
>>>>>
>>>> them
>>>>
>>>>> before doing much work to support the mocking that will be required to
>>>>>
>>>> keep
>>>>
>>>>> them inline with 'best practices'.
>>>>>
>>>>> In OFBiz "bad or unwanted" tends to come mostly in the way of increasing
>>>>> complexity from features that require more effort to maintain than the
>>>>> value they provide to the community.  I think there's a chance
>>>>> attempting
>>>>> to mock service tests could fall into that bucket.  I could be wrong,
>>>>>
>>>> but I
>>>>
>>>>> think it's worth looking into before we declare that unit tests are the
>>>>> best form of testing for OFBiz.
>>>>>
>>>>> Regards
>>>>> Scott
>>>>>
>>>>> On 19 July 2016 at 17:37, Taher Alkhateeb <[hidden email]>
>>>>> wrote:
>>>>>
>>>>> Hi Scott,
>>>>>> Thank you for the feedback. To be focused exactly on integration vs
>>>>>>
>>>>> unit, I
>>>>>
>>>>>> already mentioned above that at least for me the main objective is to
>>>>>> confidently and quickly run the tests in short bursts of red-green
>>>>>> refactor. This allows me to refactor code without waiting in front of
>>>>>>
>>>>> the
>>>>> screen in between test cycles thus giving me immediate feedback on any
>>>>>> errors I made. Perhaps my intro was too long so this is the squeezed
>>>>>>
>>>>> out
>>>>> version of it.
>>>>>> I already had one round of testing in the start component which was
>>>>>>
>>>>> much
>>>>> faster that way and had an immediate impact. Oh and by the way, you
>>>>> cannot
>>>>>
>>>>>> test the start component with integration tests for example unless you
>>>>>>
>>>>> do
>>>>> it from an external component which cannot access package protected
>>>>> items.
>>>>>
>>>>>> This style of coding is applicable I think to any software project
>>>>>> inclusive of OFBiz. Maybe in certain components more than others or
>>>>>>
>>>>> certain
>>>>>
>>>>>> areas more than others. But I can't see how it could be bad or
>>>>>>
>>>>> unwanted.
>>>>> Taher Alkhateeb
>>>>>> On Jul 19, 2016 8:18 AM, "Scott Gray" <[hidden email]>
>>>>>> wrote:
>>>>>>
>>>>>> Thanks Ron and Taher for your responses, I appreciate them but I
>>>>>> don't
>>>>> hear
>>>>>>> much in the conversation that speaks to the value of unit tests over
>>>>>>> integration tests.  Most of the thoughts shared speaks more to the
>>>>>>>
>>>>>> value
>>>>>> of
>>>>>>
>>>>>>> tests rather than differences in the type of tests.
>>>>>>>
>>>>>>> Speed: At an application level we have ~685 tests that run in 35
>>>>>>>
>>>>>> seconds
>>>>>> (excluding build and data load).  Another point is that there isn't
>>>>>> much
>>>>>> reason why tests can't be run in parallel rather than sequentially.
>>>>>>> TDD: Integration tests in OFBiz are as simple if not simpler to write
>>>>>>>
>>>>>> than
>>>>>>
>>>>>>> unit tests and just as useful for TDD.
>>>>>>> I'm not sure if I missed any other points raised regarding
>>>>>>>
>>>>>> integration
>>>>> vs.
>>>>>>> unit tests?
>>>>>>>
>>>>>>> I'm not looking to start a big long debate on the topic, I just
>>>>>>>
>>>>>> wanted
>>>>> to
>>>>>
>>>>>> speak out that someone out there (me) doesn't think unit tests are
>>>>>> the
>>>>> best
>>>>>>> solution for testing OFBiz applications.
>>>>>>>
>>>>>>> Regards
>>>>>>> Scott
>>>>>>>
>>>>>>> On 19 July 2016 at 16:52, Taher Alkhateeb <
>>>>>>>
>>>>>> [hidden email]>
>>>>> wrote:
>>>>>>> Hi Scott,
>>>>>>>> Thank you for your input. Your ideas are thought provoking and
>>>>>>>>
>>>>>>> enriching
>>>>>>> to
>>>>>>>
>>>>>>>> the conversation.
>>>>>>>>
>>>>>>>> The way I look at it, the general rule is usually many unit tests,
>>>>>>>>
>>>>>>> less
>>>>>> integration tests, lesser functional tests. So we are not excluding
>>>>>>> any
>>>>>> types of test, all of them are important in certain areas for
>>>>>>> certain
>>>>> purposes with certain quantity. Usually integration tests are less
>>>>>>> because
>>>>>>>
>>>>>>>> as you said they just grab more. I like the picture below as a
>>>>>>>>
>>>>>>> general
>>>>>> guideline
>>>>>>>> http://i.stack.imgur.com/fjQvQ.png
>>>>>>>>
>>>>>>>> As you already mentioned unit tests are useful for the framework. I
>>>>>>>> discovered errors in code that I wrote which I was very very
>>>>>>>>
>>>>>>> careful
>>>>> with.
>>>>>>>> I immediately learned the lesson that humans are not designed to
>>>>>>>>
>>>>>>> code,
>>>>>> and
>>>>>>>> TDD gives you confidence as you build your code with those short
>>>>>>>>
>>>>>>> 30-60
>>>>>> second red-green refactors. I feel much much safer and more
>>>>>>> confident
>>>>> writing code that way, the test also documents how to use the api,
>>>>>>>> refactoring and feature change becomes less terrifying. Also messy
>>>>>>>>
>>>>>>> code
>>>>>> is
>>>>>>>> usually not test friendly thus refactoring and unit tests go
>>>>>>>>
>>>>>>> hand-in-hand
>>>>>>> for improving the code base.
>>>>>>>> Also, I am sorry to say that the framework code is rather messy and
>>>>>>>>
>>>>>>> brittle
>>>>>>>
>>>>>>>> in many places. I think you probably encountered this yourself. The
>>>>>>>>
>>>>>>> same
>>>>>>> is
>>>>>>>
>>>>>>>> said for the applications. Now If we start refactoring without unit
>>>>>>>>
>>>>>>> tests
>>>>>>> then we are back to scary business. So much can go wrong so fast
>>>>>>> and
>>>>> break
>>>>>>>> things you never expected to break. The framework with all
>>>>>>>>
>>>>>>> applications
>>>>>> require heavy refactoring of things like massive ugly methods,
>>>>>>> sandwiched
>>>>>>> logic, heavy shared mutable state, hidden dependencies, poor
>>>>>>>> interfaces, and much more. Every time I touch something I get
>>>>>>>>
>>>>>>> shocked
>>>>> at
>>>>>>> how bad it looks, spaghetti logic to no end. If you refactor with
>>>>>>>> integration tests instead of unit tests then you will come down to
>>>>>>>>
>>>>>>> a
>>>>> screeching halt as you wait between these test cycles. My computer
>>>>>>> actually
>>>>>>>
>>>>>>>> takes 10 minutes or more for a full clean load data and testing.
>>>>>>>>
>>>>>>>> Now talking about mocking you raise some interesting points, what
>>>>>>>>
>>>>>>> can
>>>>> you
>>>>>>> mock vs utilizing integration tests? This is an excellent question
>>>>>>> that
>>>>>> does not have a simple answer, and you mostly learn as you code,
>>>>>>> difficult
>>>>>>>
>>>>>>>> to envision without coding. However simple things like Java
>>>>>>>>
>>>>>>> services
>>>>> can
>>>>>>> be
>>>>>>>
>>>>>>>> mocked with a standard mocking class that you use everywhere.
>>>>>>>>
>>>>>>> Transactions
>>>>>>>
>>>>>>>> are difficult to mock and better left to integration tests I think.
>>>>>>>>
>>>>>>> SECAs
>>>>>>> might be mocked by simply passing output to input as chaining
>>>>>>> methods.
>>>>>> Mind
>>>>>>>> you I am not 100% sure of all of this, coding is the ultimate guide
>>>>>>>>
>>>>>>> to
>>>>>> what
>>>>>>>> can or cannot be done.
>>>>>>>>
>>>>>>>> So I am not suggesting unit tests as a best-practice (even though
>>>>>>>>
>>>>>>> it
>>>>> is).
>>>>>>> Instead I suggest it as something that I and others did and got a
>>>>>>> huge
>>>>>> psychological relief and confidence and comfort from. Swimming in
>>>>>>> code
>>>>>> without tests is a terrifying business, made more terrifying if the
>>>>>>> code
>>>>>>> is
>>>>>>>
>>>>>>>> bad. Short bursts of red/green/red/green makes you feel good as you
>>>>>>>>
>>>>>>> build
>>>>>>> up your logic. And I don't have an exact vision of how to do it,
>>>>>>> because
>>>>>>> the details always win.
>>>>>>>> I look forward to hearing your thoughts and thank you for enriching
>>>>>>>>
>>>>>>> this
>>>>>>> conversation.
>>>>>>>> Taher Alkhateeb
>>>>>>>>
>>>>>>>> On Tuesday, 19 July 2016, Scott Gray <[hidden email]
>>>>>>>>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I know I'm late to the party here, but I just want to say that I
>>>>>>>> think
>>>>>>> integration tests have far greater value to OFBiz than unit
>>>>>>>> tests.
>>>>> Mostly
>>>>>>>>> because we tend to have quite a low number of tests and
>>>>>>>>>
>>>>>>>> integration
>>>>> tests
>>>>>>>> give us much better coverage per line of test code and the tests
>>>>>>>> are
>>>>>> much
>>>>>>>> closer to the real world scenarios the application might
>>>>>>>> encounter.
>>>>> I don't really see how unit tests could be applied to
>>>>>>>> non-framework
>>>>> testing
>>>>>>>>> in a useful manner, could you expand on your vision in that
>>>>>>>>>
>>>>>>>> regard?
>>>>> I
>>>>>>> mean
>>>>>>>>> would we be testing something smaller than a service as the
>>>>>>>>>
>>>>>>>> 'unit'?
>>>>> What
>>>>>>>> would we mock? Would transaction management still be active? What
>>>>>>>> happens
>>>>>>>> when the service calls another service, I guess we mock the
>>>>>>>> response
>>>>>> from
>>>>>>>> that service (how)?
>>>>>>>>> It just seems a very complicated method to achieve a less
>>>>>>>>>
>>>>>>>> thorough
>>>>> but
>>>>>>> albeit faster (maybe) test result.
>>>>>>>>> A build, data load and full test run takes 4m 9s on my laptop.
>>>>>>>>>
>>>>>>>> Excluding
>>>>>>>> build, data load and framework tests: the application level tests
>>>>>>>> take
>>>>>>> 35s,
>>>>>>>>> not very expensive IMO.  The data load time can be reduced to
>>>>>>>>>
>>>>>>>> practically
>>>>>>>> nothing by copying a clean slate database into runtime for each
>>>>>>>> run.
>>>>>> I'm mostly just suggesting we be wary of adding complicated
>>>>>>>> testing
>>>>> procedures in the hope of achieving some 'best practice' result
>>>>>>>> which
>>>>>> in
>>>>>>>> reality will provide minimal benefits.
>>>>>>>>> Regards
>>>>>>>>> Scott
>>>>>>>>>
>>>>>>>>> On 18 July 2016 at 18:57, Taher Alkhateeb <
>>>>>>>>>
>>>>>>>> [hidden email]
>>>>>>> <javascript:;>>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Hello Akash,
>>>>>>>>>> Fantastic, I have a few unit tests almost done to be included
>>>>>>>>>>
>>>>>>>>> in
>>>>> the
>>>>>>> start
>>>>>>>>>> component. I will create a new subtask under OFBIZ-1463 to
>>>>>>>>>>
>>>>>>>>> commit
>>>>> the
>>>>>>> tests
>>>>>>>>>> so you can use them as a reference if you like to.
>>>>>>>>>>
>>>>>>>>>> I also recommend that you follow the same directory structure
>>>>>>>>>>
>>>>>>>>> between
>>>>>>> the
>>>>>>>>> test code and production code. So for example:
>>>>>>>>>> Production code:
>>>>>>>>>>
>>>>>>>>> framework/start/src/main/java/org/apache/ofbiz/base/start
>>>>>>>>>
>>>>>>>>>> Test code:
>>>>>>>>>>
>>>>>>>>> framework/start/src/test/java/org/apache/ofbiz/base/start
>>>>>>> The benefit of this hierarchy is that you can access non-public
>>>>>>>>> (package
>>>>>>>>> protected) methods for testing. This is in fact exactly what I
>>>>>>>>> needed
>>>>>>> to
>>>>>>>>> be
>>>>>>>>>
>>>>>>>>>> able to apply some of the tests.
>>>>>>>>>>
>>>>>>>>>> Cheers,
>>>>>>>>>>
>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>
>>>>>>>>>> On Mon, Jul 18, 2016 at 9:22 AM, Akash Jain <
>>>>>>>>>>
>>>>>>>>> [hidden email] <javascript:;>>
>>>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Thanks Taher for nice initiative!
>>>>>>>>>>> We are planning to written unit tests to all components under
>>>>>>>>>>>
>>>>>>>>>> OFBIZ-1463
>>>>>>>>>> Thanks and Regards
>>>>>>>>>>> --
>>>>>>>>>>> Akash Jain
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Jul 18, 2016 at 10:36 AM, Taher Alkhateeb <
>>>>>>>>>>> [hidden email] <javascript:;>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Hello Everyone,
>>>>>>>>>>>> In reference to this thread and the Jira OFBIZ-7254, I'm
>>>>>>>>>>>>
>>>>>>>>>>> very
>>>>> happy
>>>>>>>> to
>>>>>>>>>> announce that OFBiz is now ready for applying unit tests
>>>>>>>>>>> with
>>>>> only
>>>>>>>> 8
>>>>>>>>
>>>>>>>>> new
>>>>>>>>>>> lines of code in the build script (r1753143) :)
>>>>>>>>>>>> I recommend we do the following moving forward:
>>>>>>>>>>>>
>>>>>>>>>>>> 1- Introduce unit tests as much as we can to all components
>>>>>>>>>>>> 2- Migrate most of the integration tests we currently have
>>>>>>>>>>>>
>>>>>>>>>>> to
>>>>> unit
>>>>>>>> tests
>>>>>>>>>>> (since they are designed to do very little integration).
>>>>>>>>>>>> This is a great chance for us to practice real TDD in OFBiz
>>>>>>>>>>>>
>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>
>>>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Jun 13, 2016 at 7:43 AM, Pranay Pandey <
>>>>>>>>>>>> [hidden email] <javascript:;>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> +1
>>>>>>>>>>>>> Best regards,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Pranay Pandey
>>>>>>>>>>>>> HotWax Systems
>>>>>>>>>>>>> http://www.hotwaxsystems.com/
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Jun 10, 2016 at 7:46 PM, Taher Alkhateeb <
>>>>>>>>>>>>> [hidden email] <javascript:;>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> Hello Everyone,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I was able to get a few tests running and this is very
>>>>>>>>>>>>>>
>>>>>>>>>>>>> doable.
>>>>>>>> But
>>>>>>>>>> I
>>>>>>>>>>
>>>>>>>>>>> faced
>>>>>>>>>>>>>> a big problem in designing the testing framework
>>>>>>>>>>>>>>
>>>>>>>>>>>>> because
>>>>> of
>>>>>
>>>>>> ANT.
>>>>>>>>> The problem
>>>>>>>>>>>>>> ----------------
>>>>>>>>>>>>>> The way the build scripts are designed in OFBiz are
>>>>>>>>>>>>>>
>>>>>>>>>>>>> very
>>>>> complex. A
>>>>>>>>>> master
>>>>>>>>>>>>>> file calls other files which call other files. And in
>>>>>>>>>>>>>>
>>>>>>>>>>>>> the
>>>>> middle
>>>>>>>>> you
>>>>>>>>>>> have
>>>>>>>>>>>>> external libraries (ant-contrib) and macros, and
>>>>>>>>>>>>> variables,
>>>>>> and
>>>>>>>> class
>>>>>>>>>>> path
>>>>>>>>>>>>>> declarations, and and and ....
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I cannot declare the tests programmatically (with JUnit
>>>>>>>>>>>>>>
>>>>>>>>>>>>> test
>>>>>>> suites)
>>>>>>>>>>> because this means lower level components would depend
>>>>>>>>>>>>> on
>>>>> higher
>>>>>>>>> level
>>>>>>>>>>>> components. So I have to do it in ANT, by navigating
>>>>>>>>>>>>> this
>>>>> maze
>>>>>>>> of
>>>>>>>>
>>>>>>>>> build
>>>>>>>>>>>> scripts, and it was a headache for me just to read
>>>>>>>>>>>>> them,
>>>>> let
>>>>>>> alone
>>>>>>>>>> modify
>>>>>>>>>>>>> them to create a testing framework.
>>>>>>>>>>>>>> Suggested Solution
>>>>>>>>>>>>>> ------------------------
>>>>>>>>>>>>>> I suggest to implement the testing framework in Gradle,
>>>>>>>>>>>>>>
>>>>>>>>>>>>> and
>>>>>> simply
>>>>>>>>>> call
>>>>>>>>>>>> it
>>>>>>>>>>>>>> from within ant. This is a middle solution that
>>>>>>>>>>>>>>
>>>>>>>>>>>>> sustains
>>>>> ant
>>>>>>> for
>>>>>>>>> now,
>>>>>>>>>>> but
>>>>>>>>>>>>> can allow us to switch out later.
>>>>>>>>>>>>>> This means I will just add one more file called
>>>>>>>>>>>>>>
>>>>>>>>>>>>> build.gradle
>>>>>>> in
>>>>>>>
>>>>>>>> the
>>>>>>>>>> top
>>>>>>>>>>>> level directory, and figure out the business logic for
>>>>>>>>>>>>> calling
>>>>>>>> the
>>>>>>>>>> test
>>>>>>>>>>>> suites from that file
>>>>>>>>>>>>>> I look forward to your feedback.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <
>>>>>>>>>>>>>> [hidden email] <javascript:;>>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Everyone,
>>>>>>>>>>>>>>> Thank you all for your support, JIRA created in
>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OFBIZ-7254
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I will start working on it and try to implement ASAP
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> to
>>>>> get
>>>>>>> my
>>>>>>>>> focus
>>>>>>>>>>>> back
>>>>>>>>>>>>>> on refactoring.
>>>>>>>>>>>>>>> Cheers!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
>>>>>>>>>>>>>>> [hidden email] <javascript:;>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> wrote:
>>>>> +1
>>>>>>>>>>>>>>>> Thanks & Regards
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> Deepak Dixit
>>>>>>>>>>>>>>>> www.hotwaxsystems.com
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
>>>>>>>>>>>>>>>> [hidden email] <javascript:;>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> wrote:
>>>>>> +1
>>>>>>>>>>>>>>>>> Makes perfect sense.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> Thanks & Regards,
>>>>>>>>>>>>>>>>> Mridul Pathak
>>>>>>>>>>>>>>>>> Senior Manager
>>>>>>>>>>>>>>>>> HotWax Systems
>>>>>>>>>>>>>>>>> http://www.hotwaxsystems.com
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
>>>>>>>>>>>>>>>>> [hidden email] <javascript:;>>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hello Everyone,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> After refactoring the start component and while
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> starting
>>>>>>>> on
>>>>>>>>>> the
>>>>>>>>>>>> base
>>>>>>>>>>>>>> component I realized that the testing framework
>>>>>>>>>>>>>>>>> for
>>>>>> OFBiz
>>>>>>>>> is
>>>>>>>>>
>>>>>>>>>> not
>>>>>>>>>>>> good.
>>>>>>>>>>>>>>> You
>>>>>>>>>>>>>>>>>> cannot do real test driven development or
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> red-green-refactor
>>>>>>>>>> with
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> current setup, hence my proposal to change it. I
>>>>>>>>>>>>>>>>> explain
>>>>>>>> below:
>>>>>>>>>>>> Problem with current design
>>>>>>>>>>>>>>>>>> ----------------------------------------
>>>>>>>>>>>>>>>>>> - What we have right now is not unit tests, it's
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> really
>>>>>>>> integration
>>>>>>>>>>>>>> tests.
>>>>>>>>>>>>>>>>>> You have to start the framework, the database,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> the
>>>>> service
>>>>>>>>> engine,
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> entity engine and pretty much everything.
>>>>>>>>>>>>>>>>>> - Testing is very slow, because it's an
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> integration
>>>>>> test
>>>>>>>> as
>>>>>>>>>> I
>>>>>>>>>>
>>>>>>>>>>> mentioned
>>>>>>>>>>>>>>>>> above. 10 minutes on a good computer!
>>>>>>>>>>>>>>>>>> - There is zero mocking! We actually have to
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --load-data
>>>>>>>> for
>>>>>>>>>> things
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> work. Again, these are integration tests.
>>>>>>>>>>>>>>>>>> - Too complex: Integration tests by their nature
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> are
>>>>>> grabbing
>>>>>>>>>>> too
>>>>>>>>>>>>> much.
>>>>>>>>>>>>>>>>> Mind you, I am not objecting to integration
>>>>>>>>>>>>>>>>> tests
>>>>> (I
>>>>>
>>>>>> actually
>>>>>>>>>>> like
>>>>>>>>>>>>> them)
>>>>>>>>>>>>>>>>> but I am objecting to not having real
>>>>>>>>>>>>>>>>> unit-tests.
>>>>> Unit
>>>>>>> tests
>>>>>>>>>> should
>>>>>>>>>>>>>> all
>>>>>>>>>>>>>>>>> run
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> in a few seconds.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Proposed solution
>>>>>>>>>>>>>>>>>> --------------------------
>>>>>>>>>>>>>>>>>> - We keep what is considered real integration
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> tests
>>>>>> the
>>>>>>>> way
>>>>>>>>>> they
>>>>>>>>>>>> are
>>>>>>>>>>>>>> right
>>>>>>>>>>>>>>>>>> now and keep using them
>>>>>>>>>>>>>>>>>> - We move what should be unit tests into simple
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> JUnit
>>>>>>> classes,
>>>>>>>>>>> and
>>>>>>>>>>>>> we
>>>>>>>>>>>>>>> do
>>>>>>>>>>>>>>>>> not run them using java -jar ofbiz.jar --test,
>>>>>>>>>>>>>>>>> but
>>>>> instead
>>>>>>>>> run
>>>>>>>>>>> them
>>>>>>>>>>>>>> directly from the build.xml script, so these
>>>>>>>>>>>>>>>>> files
>>>>> are
>>>>>>> not
>>>>>>>>> identified
>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>> any XML document, but are simply called
>>>>>>>>>>>>>>>>> immediately
>>>>>> from
>>>>>>>> the
>>>>>>>>>> build
>>>>>>>>>>>>> scripts.
>>>>>>>>>>>>>>>>>> - We clearly mark the difference between
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> integration
>>>>>> tests
>>>>>>>>> and
>>>>>>>>>>> unit
>>>>>>>>>>>>>> tests
>>>>>>>>>>>>>>>>> (inside the source files or in the suite
>>>>>>>>>>>>>>>>> declarations).
>>>>>>>> - We change the run-tests target in build.xml to
>>>>>>>>>>>>>>>>> run
>>>>>> both
>>>>>>>>> unit
>>>>>>>>>>> tests
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> integration tests.
>>>>>>>>>>>>>>>>>> I intend to heavily refactor the framework and I
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> would
>>>>>>> feel
>>>>>>>>>> better
>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>> introducing this change while refactoring. What
>>>>>>>>>>>>>>>>> do
>>>>> you
>>>>>>> guys
>>>>>>>>>> think?
>>>>>>>>>>>>> Ideas?
>>>>>>>>>>>>>>>>> Suggestions? Approvals and thumbs up?
>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>

--
Ron Wheeler
President
Artifact Software Inc
email: [hidden email]
skype: ronaldmwheeler
phone: [phone], ext 102

[Scott Gray-3]

I'm not looking for any sort of response really.  I was just speaking in
favor of integration tests because I think they provide better coverage at
the cost of a little bit of speed (the framework has to be running to run
them).  Most of the responses coming in were of the form "unit tests are
great because [insert a reason that applies to any type of test]".  So I
was merely pointing out that the arguments weren't specific to unit tests
and hence weren't really adding value to the discussion.

Regards
Scott

On 20 July 2016 at 02:05, Ron Wheeler <[hidden email]>
wrote:

> It is not clear what you are expecting as a specific response.
>
> As you move up the food chain, the definition of "units" changes and the
> tests get more complicated and the design of the tests gets more complex
> (mock objects, testing race conditions).
>
> It is hard to make specific comments about what kind of testing will give
> the highest level of confidence at each level but we are going in the right
> direction.
>
> The fact that we are considering tests as a major development activity is
> a big step forward.
>
> We can argue about specific cases (scope, tools, value, etc) as we get to
> them.
>
> I think that we are at the "good idea" stage rather than at the "policy"
> stage at this point.
>
> Ron
>
>
>
> On 19/07/2016 4:23 AM, Scott Gray wrote:
>
>> I think people are missing my point because they keep replying with
>> generic
>> statements that aren't specific to unit tests.
>>
>> On 19 July 2016 at 20:08, Montalbano Florian <
>> [hidden email]>
>> wrote:
>>
>> Hi everyone,
>>>
>>> unit tests and integration tests are complementary and with test, the
>>> more
>>> the better.
>>> Implementing unit tests will be a great load of work but it is the same
>>> with every kind of test. As Taher said, we will gain a lot from those
>>> unit
>>> tests for avoiding regression while refactoring the framework. From the
>>> start component scope, it has already be proved useful.
>>> And what Ron said about encouraging the creation of "smaller class that
>>> can be tested easily" is very important too. It can be a first step to
>>> reach TDD in the OFBiz development. TDD can help to give confidence to
>>> newcomer and veteran when updating some part of the code and as Hans
>>> said,
>>> this lead to "continuous improvement and finally continuous deployment".
>>>
>>> I'm aware that this does not solve the technical implementation stuff,
>>> but
>>> I think this is an improvement that need to be supported.
>>>
>>> Have a nice day,
>>>
>>> Florian Montalbano
>>>
>>>
>>> Le 19/07/2016 09:04, Pierre Smits a écrit :
>>>
>>> As with anything, the law of diminishing returns also applies to OFBiz
>>>> and
>>>> tests. This is not true for unit tests and system integration tests, but
>>>> also for user acceptance and performance tests.
>>>>
>>>> Nevertheless, the work done up to now is a good start and - I feel
>>>> confident - appreciated. And unit tests are certainly valuable in the
>>>> framework stack. How it will be for functions (regarding components in
>>>> application and special purpose stack) needs to be addressed when we
>>>> reach
>>>> that bridge.
>>>>
>>>>
>>>> Best regards,
>>>>
>>>> Pierre Smits
>>>>
>>>> ORRTIZ.COM <http://www.orrtiz.com>
>>>> OFBiz based solutions & services
>>>>
>>>> OFBiz Extensions Marketplace
>>>> http://oem.ofbizci.net/oci-2/
>>>>
>>>> On Tue, Jul 19, 2016 at 8:22 AM, Taher Alkhateeb <
>>>> [hidden email]
>>>>
>>>> wrote:
>>>>> Hi Scott,
>>>>>
>>>>> Well spoken thank you. Okay may I suggest that for any such work we
>>>>> will
>>>>> discuss it here see its Merit and if it makes sense then we take it in.
>>>>> It
>>>>> is a little early to discuss it right now because we did not yet go to
>>>>> the
>>>>> higher-level components. Once we do I'll be sure to have a conversation
>>>>> about this in here and would appreciate your input to it.
>>>>>
>>>>> Regards,
>>>>>
>>>>> Taher Alkhateeb
>>>>>
>>>>> On Jul 19, 2016 9:00 AM, "Scott Gray" <[hidden email]>
>>>>> wrote:
>>>>>
>>>>> Yeah I'm sure unit tests probably worked well for the start component
>>>>> given
>>>>>
>>>>> it is the lowest level component in the system and closest to a basic
>>>>>>
>>>>>> java
>>>>>
>>>>> app.  I just think the value proposition might decrease the further up
>>>>>>
>>>>>> the
>>>>>
>>>>> stack we move with them.  I'm not against unit tests when they prove
>>>>>> useful, but further up the stack I think we should prove the case for
>>>>>>
>>>>>> them
>>>>>
>>>>> before doing much work to support the mocking that will be required to
>>>>>>
>>>>>> keep
>>>>>
>>>>> them inline with 'best practices'.
>>>>>>
>>>>>> In OFBiz "bad or unwanted" tends to come mostly in the way of
>>>>>> increasing
>>>>>> complexity from features that require more effort to maintain than the
>>>>>> value they provide to the community.  I think there's a chance
>>>>>> attempting
>>>>>> to mock service tests could fall into that bucket.  I could be wrong,
>>>>>>
>>>>>> but I
>>>>>
>>>>> think it's worth looking into before we declare that unit tests are the
>>>>>> best form of testing for OFBiz.
>>>>>>
>>>>>> Regards
>>>>>> Scott
>>>>>>
>>>>>> On 19 July 2016 at 17:37, Taher Alkhateeb <[hidden email]
>>>>>> >
>>>>>> wrote:
>>>>>>
>>>>>> Hi Scott,
>>>>>>
>>>>>>> Thank you for the feedback. To be focused exactly on integration vs
>>>>>>>
>>>>>>> unit, I
>>>>>>
>>>>>> already mentioned above that at least for me the main objective is to
>>>>>>> confidently and quickly run the tests in short bursts of red-green
>>>>>>> refactor. This allows me to refactor code without waiting in front of
>>>>>>>
>>>>>>> the
>>>>>> screen in between test cycles thus giving me immediate feedback on any
>>>>>>
>>>>>>> errors I made. Perhaps my intro was too long so this is the squeezed
>>>>>>>
>>>>>>> out
>>>>>> version of it.
>>>>>>
>>>>>>> I already had one round of testing in the start component which was
>>>>>>>
>>>>>>> much
>>>>>> faster that way and had an immediate impact. Oh and by the way, you
>>>>>> cannot
>>>>>>
>>>>>> test the start component with integration tests for example unless you
>>>>>>>
>>>>>>> do
>>>>>> it from an external component which cannot access package protected
>>>>>> items.
>>>>>>
>>>>>> This style of coding is applicable I think to any software project
>>>>>>> inclusive of OFBiz. Maybe in certain components more than others or
>>>>>>>
>>>>>>> certain
>>>>>>
>>>>>> areas more than others. But I can't see how it could be bad or
>>>>>>>
>>>>>>> unwanted.
>>>>>> Taher Alkhateeb
>>>>>>
>>>>>>> On Jul 19, 2016 8:18 AM, "Scott Gray" <[hidden email]>
>>>>>>> wrote:
>>>>>>>
>>>>>>> Thanks Ron and Taher for your responses, I appreciate them but I
>>>>>>> don't
>>>>>>>
>>>>>> hear
>>>>>>
>>>>>>> much in the conversation that speaks to the value of unit tests over
>>>>>>>> integration tests.  Most of the thoughts shared speaks more to the
>>>>>>>>
>>>>>>>> value
>>>>>>> of
>>>>>>>
>>>>>>> tests rather than differences in the type of tests.
>>>>>>>>
>>>>>>>> Speed: At an application level we have ~685 tests that run in 35
>>>>>>>>
>>>>>>>> seconds
>>>>>>> (excluding build and data load).  Another point is that there isn't
>>>>>>> much
>>>>>>> reason why tests can't be run in parallel rather than sequentially.
>>>>>>>
>>>>>>>> TDD: Integration tests in OFBiz are as simple if not simpler to
>>>>>>>> write
>>>>>>>>
>>>>>>>> than
>>>>>>>
>>>>>>> unit tests and just as useful for TDD.
>>>>>>>> I'm not sure if I missed any other points raised regarding
>>>>>>>>
>>>>>>>> integration
>>>>>>>
>>>>>> vs.
>>>>>>
>>>>>>> unit tests?
>>>>>>>>
>>>>>>>> I'm not looking to start a big long debate on the topic, I just
>>>>>>>>
>>>>>>>> wanted
>>>>>>>
>>>>>> to
>>>>>>
>>>>>> speak out that someone out there (me) doesn't think unit tests are
>>>>>>> the
>>>>>>>
>>>>>> best
>>>>>>
>>>>>>> solution for testing OFBiz applications.
>>>>>>>>
>>>>>>>> Regards
>>>>>>>> Scott
>>>>>>>>
>>>>>>>> On 19 July 2016 at 16:52, Taher Alkhateeb <
>>>>>>>>
>>>>>>>> [hidden email]>
>>>>>>>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Scott,
>>>>>>>>
>>>>>>>>> Thank you for your input. Your ideas are thought provoking and
>>>>>>>>>
>>>>>>>>> enriching
>>>>>>>> to
>>>>>>>>
>>>>>>>> the conversation.
>>>>>>>>>
>>>>>>>>> The way I look at it, the general rule is usually many unit tests,
>>>>>>>>>
>>>>>>>>> less
>>>>>>>>
>>>>>>> integration tests, lesser functional tests. So we are not excluding
>>>>>>>
>>>>>>>> any
>>>>>>>>
>>>>>>> types of test, all of them are important in certain areas for
>>>>>>>
>>>>>>>> certain
>>>>>>>>
>>>>>>> purposes with certain quantity. Usually integration tests are less
>>>>>>
>>>>>>> because
>>>>>>>>
>>>>>>>> as you said they just grab more. I like the picture below as a
>>>>>>>>>
>>>>>>>>> general
>>>>>>>>
>>>>>>> guideline
>>>>>>>
>>>>>>>> http://i.stack.imgur.com/fjQvQ.png
>>>>>>>>>
>>>>>>>>> As you already mentioned unit tests are useful for the framework. I
>>>>>>>>> discovered errors in code that I wrote which I was very very
>>>>>>>>>
>>>>>>>>> careful
>>>>>>>>
>>>>>>> with.
>>>>>>
>>>>>>> I immediately learned the lesson that humans are not designed to
>>>>>>>>>
>>>>>>>>> code,
>>>>>>>>
>>>>>>> and
>>>>>>>
>>>>>>>> TDD gives you confidence as you build your code with those short
>>>>>>>>>
>>>>>>>>> 30-60
>>>>>>>>
>>>>>>> second red-green refactors. I feel much much safer and more
>>>>>>>
>>>>>>>> confident
>>>>>>>>
>>>>>>> writing code that way, the test also documents how to use the api,
>>>>>>
>>>>>>> refactoring and feature change becomes less terrifying. Also messy
>>>>>>>>>
>>>>>>>>> code
>>>>>>>>
>>>>>>> is
>>>>>>>
>>>>>>>> usually not test friendly thus refactoring and unit tests go
>>>>>>>>>
>>>>>>>>> hand-in-hand
>>>>>>>> for improving the code base.
>>>>>>>>
>>>>>>>>> Also, I am sorry to say that the framework code is rather messy and
>>>>>>>>>
>>>>>>>>> brittle
>>>>>>>>
>>>>>>>> in many places. I think you probably encountered this yourself. The
>>>>>>>>>
>>>>>>>>> same
>>>>>>>> is
>>>>>>>>
>>>>>>>> said for the applications. Now If we start refactoring without unit
>>>>>>>>>
>>>>>>>>> tests
>>>>>>>> then we are back to scary business. So much can go wrong so fast
>>>>>>>> and
>>>>>>>>
>>>>>>> break
>>>>>>
>>>>>>> things you never expected to break. The framework with all
>>>>>>>>>
>>>>>>>>> applications
>>>>>>>>
>>>>>>> require heavy refactoring of things like massive ugly methods,
>>>>>>>
>>>>>>>> sandwiched
>>>>>>>> logic, heavy shared mutable state, hidden dependencies, poor
>>>>>>>>
>>>>>>>>> interfaces, and much more. Every time I touch something I get
>>>>>>>>>
>>>>>>>>> shocked
>>>>>>>>
>>>>>>> at
>>>>>>
>>>>>>> how bad it looks, spaghetti logic to no end. If you refactor with
>>>>>>>>
>>>>>>>>> integration tests instead of unit tests then you will come down to
>>>>>>>>>
>>>>>>>>> a
>>>>>>>>
>>>>>>> screeching halt as you wait between these test cycles. My computer
>>>>>>
>>>>>>> actually
>>>>>>>>
>>>>>>>> takes 10 minutes or more for a full clean load data and testing.
>>>>>>>>>
>>>>>>>>> Now talking about mocking you raise some interesting points, what
>>>>>>>>>
>>>>>>>>> can
>>>>>>>>
>>>>>>> you
>>>>>>
>>>>>>> mock vs utilizing integration tests? This is an excellent question
>>>>>>>> that
>>>>>>>>
>>>>>>> does not have a simple answer, and you mostly learn as you code,
>>>>>>>
>>>>>>>> difficult
>>>>>>>>
>>>>>>>> to envision without coding. However simple things like Java
>>>>>>>>>
>>>>>>>>> services
>>>>>>>>
>>>>>>> can
>>>>>>
>>>>>>> be
>>>>>>>>
>>>>>>>> mocked with a standard mocking class that you use everywhere.
>>>>>>>>>
>>>>>>>>> Transactions
>>>>>>>>
>>>>>>>> are difficult to mock and better left to integration tests I think.
>>>>>>>>>
>>>>>>>>> SECAs
>>>>>>>> might be mocked by simply passing output to input as chaining
>>>>>>>> methods.
>>>>>>>>
>>>>>>> Mind
>>>>>>>
>>>>>>>> you I am not 100% sure of all of this, coding is the ultimate guide
>>>>>>>>>
>>>>>>>>> to
>>>>>>>>
>>>>>>> what
>>>>>>>
>>>>>>>> can or cannot be done.
>>>>>>>>>
>>>>>>>>> So I am not suggesting unit tests as a best-practice (even though
>>>>>>>>>
>>>>>>>>> it
>>>>>>>>
>>>>>>> is).
>>>>>>
>>>>>>> Instead I suggest it as something that I and others did and got a
>>>>>>>> huge
>>>>>>>>
>>>>>>> psychological relief and confidence and comfort from. Swimming in
>>>>>>>
>>>>>>>> code
>>>>>>>>
>>>>>>> without tests is a terrifying business, made more terrifying if the
>>>>>>>
>>>>>>>> code
>>>>>>>> is
>>>>>>>>
>>>>>>>> bad. Short bursts of red/green/red/green makes you feel good as you
>>>>>>>>>
>>>>>>>>> build
>>>>>>>> up your logic. And I don't have an exact vision of how to do it,
>>>>>>>> because
>>>>>>>> the details always win.
>>>>>>>>
>>>>>>>>> I look forward to hearing your thoughts and thank you for enriching
>>>>>>>>>
>>>>>>>>> this
>>>>>>>> conversation.
>>>>>>>>
>>>>>>>>> Taher Alkhateeb
>>>>>>>>>
>>>>>>>>> On Tuesday, 19 July 2016, Scott Gray <[hidden email]
>>>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> I know I'm late to the party here, but I just want to say that I
>>>>>>>>> think
>>>>>>>>>
>>>>>>>> integration tests have far greater value to OFBiz than unit
>>>>>>>>
>>>>>>>>> tests.
>>>>>>>>>
>>>>>>>> Mostly
>>>>>>
>>>>>>> because we tend to have quite a low number of tests and
>>>>>>>>>>
>>>>>>>>>> integration
>>>>>>>>>
>>>>>>>> tests
>>>>>>
>>>>>>> give us much better coverage per line of test code and the tests
>>>>>>>>> are
>>>>>>>>>
>>>>>>>> much
>>>>>>>
>>>>>>>> closer to the real world scenarios the application might
>>>>>>>>> encounter.
>>>>>>>>>
>>>>>>>> I don't really see how unit tests could be applied to
>>>>>>
>>>>>>> non-framework
>>>>>>>>>
>>>>>>>> testing
>>>>>>
>>>>>>> in a useful manner, could you expand on your vision in that
>>>>>>>>>>
>>>>>>>>>> regard?
>>>>>>>>>
>>>>>>>> I
>>>>>>
>>>>>>> mean
>>>>>>>>
>>>>>>>>> would we be testing something smaller than a service as the
>>>>>>>>>>
>>>>>>>>>> 'unit'?
>>>>>>>>>
>>>>>>>> What
>>>>>>
>>>>>>> would we mock? Would transaction management still be active? What
>>>>>>>>> happens
>>>>>>>>> when the service calls another service, I guess we mock the
>>>>>>>>> response
>>>>>>>>>
>>>>>>>> from
>>>>>>>
>>>>>>>> that service (how)?
>>>>>>>>>
>>>>>>>>>> It just seems a very complicated method to achieve a less
>>>>>>>>>>
>>>>>>>>>> thorough
>>>>>>>>>
>>>>>>>> but
>>>>>>
>>>>>>> albeit faster (maybe) test result.
>>>>>>>>
>>>>>>>>> A build, data load and full test run takes 4m 9s on my laptop.
>>>>>>>>>>
>>>>>>>>>> Excluding
>>>>>>>>> build, data load and framework tests: the application level tests
>>>>>>>>> take
>>>>>>>>>
>>>>>>>> 35s,
>>>>>>>>
>>>>>>>>> not very expensive IMO.  The data load time can be reduced to
>>>>>>>>>>
>>>>>>>>>> practically
>>>>>>>>> nothing by copying a clean slate database into runtime for each
>>>>>>>>> run.
>>>>>>>>>
>>>>>>>> I'm mostly just suggesting we be wary of adding complicated
>>>>>>>
>>>>>>>> testing
>>>>>>>>>
>>>>>>>> procedures in the hope of achieving some 'best practice' result
>>>>>>
>>>>>>> which
>>>>>>>>>
>>>>>>>> in
>>>>>>>
>>>>>>>> reality will provide minimal benefits.
>>>>>>>>>
>>>>>>>>>> Regards
>>>>>>>>>> Scott
>>>>>>>>>>
>>>>>>>>>> On 18 July 2016 at 18:57, Taher Alkhateeb <
>>>>>>>>>>
>>>>>>>>>> [hidden email]
>>>>>>>>>
>>>>>>>> <javascript:;>>
>>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Hello Akash,
>>>>>>>>>>
>>>>>>>>>>> Fantastic, I have a few unit tests almost done to be included
>>>>>>>>>>>
>>>>>>>>>>> in
>>>>>>>>>>
>>>>>>>>> the
>>>>>>
>>>>>>> start
>>>>>>>>
>>>>>>>>> component. I will create a new subtask under OFBIZ-1463 to
>>>>>>>>>>>
>>>>>>>>>>> commit
>>>>>>>>>>
>>>>>>>>> the
>>>>>>
>>>>>>> tests
>>>>>>>>
>>>>>>>>> so you can use them as a reference if you like to.
>>>>>>>>>>>
>>>>>>>>>>> I also recommend that you follow the same directory structure
>>>>>>>>>>>
>>>>>>>>>>> between
>>>>>>>>>>
>>>>>>>>> the
>>>>>>>>
>>>>>>>>> test code and production code. So for example:
>>>>>>>>>>
>>>>>>>>>>> Production code:
>>>>>>>>>>>
>>>>>>>>>>> framework/start/src/main/java/org/apache/ofbiz/base/start
>>>>>>>>>>
>>>>>>>>>> Test code:
>>>>>>>>>>>
>>>>>>>>>>> framework/start/src/test/java/org/apache/ofbiz/base/start
>>>>>>>>>>
>>>>>>>>> The benefit of this hierarchy is that you can access non-public
>>>>>>>>
>>>>>>>>> (package
>>>>>>>>>> protected) methods for testing. This is in fact exactly what I
>>>>>>>>>> needed
>>>>>>>>>>
>>>>>>>>> to
>>>>>>>>
>>>>>>>>> be
>>>>>>>>>>
>>>>>>>>>> able to apply some of the tests.
>>>>>>>>>>>
>>>>>>>>>>> Cheers,
>>>>>>>>>>>
>>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Jul 18, 2016 at 9:22 AM, Akash Jain <
>>>>>>>>>>>
>>>>>>>>>>> [hidden email] <javascript:;>>
>>>>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Thanks Taher for nice initiative!
>>>>>>>>>>>
>>>>>>>>>>>> We are planning to written unit tests to all components under
>>>>>>>>>>>>
>>>>>>>>>>>> OFBIZ-1463
>>>>>>>>>>> Thanks and Regards
>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Akash Jain
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Jul 18, 2016 at 10:36 AM, Taher Alkhateeb <
>>>>>>>>>>>> [hidden email] <javascript:;>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Hello Everyone,
>>>>>>>>>>>>
>>>>>>>>>>>>> In reference to this thread and the Jira OFBIZ-7254, I'm
>>>>>>>>>>>>>
>>>>>>>>>>>>> very
>>>>>>>>>>>>
>>>>>>>>>>> happy
>>>>>>
>>>>>>> to
>>>>>>>>>
>>>>>>>>>> announce that OFBiz is now ready for applying unit tests
>>>>>>>>>>>
>>>>>>>>>>>> with
>>>>>>>>>>>>
>>>>>>>>>>> only
>>>>>>
>>>>>>> 8
>>>>>>>>>
>>>>>>>>> new
>>>>>>>>>>
>>>>>>>>>>> lines of code in the build script (r1753143) :)
>>>>>>>>>>>>
>>>>>>>>>>>>> I recommend we do the following moving forward:
>>>>>>>>>>>>>
>>>>>>>>>>>>> 1- Introduce unit tests as much as we can to all components
>>>>>>>>>>>>> 2- Migrate most of the integration tests we currently have
>>>>>>>>>>>>>
>>>>>>>>>>>>> to
>>>>>>>>>>>>
>>>>>>>>>>> unit
>>>>>>
>>>>>>> tests
>>>>>>>>>
>>>>>>>>>> (since they are designed to do very little integration).
>>>>>>>>>>>>
>>>>>>>>>>>>> This is a great chance for us to practice real TDD in OFBiz
>>>>>>>>>>>>>
>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Jun 13, 2016 at 7:43 AM, Pranay Pandey <
>>>>>>>>>>>>> [hidden email] <javascript:;>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> +1
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Best regards,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Pranay Pandey
>>>>>>>>>>>>>> HotWax Systems
>>>>>>>>>>>>>> http://www.hotwaxsystems.com/
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Fri, Jun 10, 2016 at 7:46 PM, Taher Alkhateeb <
>>>>>>>>>>>>>> [hidden email] <javascript:;>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> Hello Everyone,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I was able to get a few tests running and this is very
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> doable.
>>>>>>>>>>>>>>
>>>>>>>>>>>>> But
>>>>>>>>>
>>>>>>>>>> I
>>>>>>>>>>>
>>>>>>>>>>> faced
>>>>>>>>>>>>
>>>>>>>>>>>>> a big problem in designing the testing framework
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> because
>>>>>>>>>>>>>>
>>>>>>>>>>>>> of
>>>>>>
>>>>>> ANT.
>>>>>>>
>>>>>>>> The problem
>>>>>>>>>>
>>>>>>>>>>> ----------------
>>>>>>>>>>>>>>> The way the build scripts are designed in OFBiz are
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> very
>>>>>>>>>>>>>>
>>>>>>>>>>>>> complex. A
>>>>>>
>>>>>>> master
>>>>>>>>>>>
>>>>>>>>>>>> file calls other files which call other files. And in
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>
>>>>>>>>>>>>> middle
>>>>>>
>>>>>>> you
>>>>>>>>>>
>>>>>>>>>>> have
>>>>>>>>>>>>
>>>>>>>>>>>>> external libraries (ant-contrib) and macros, and
>>>>>>>>>>>>>> variables,
>>>>>>>>>>>>>>
>>>>>>>>>>>>> and
>>>>>>>
>>>>>>>> class
>>>>>>>>>
>>>>>>>>>> path
>>>>>>>>>>>>
>>>>>>>>>>>>> declarations, and and and ....
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I cannot declare the tests programmatically (with JUnit
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> test
>>>>>>>>>>>>>>
>>>>>>>>>>>>> suites)
>>>>>>>>
>>>>>>>>> because this means lower level components would depend
>>>>>>>>>>>>
>>>>>>>>>>>>> on
>>>>>>>>>>>>>>
>>>>>>>>>>>>> higher
>>>>>>
>>>>>>> level
>>>>>>>>>>
>>>>>>>>>>> components. So I have to do it in ANT, by navigating
>>>>>>>>>>>>>
>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>
>>>>>>>>>>>>> maze
>>>>>>
>>>>>>> of
>>>>>>>>>
>>>>>>>>> build
>>>>>>>>>>
>>>>>>>>>>> scripts, and it was a headache for me just to read
>>>>>>>>>>>>>
>>>>>>>>>>>>>> them,
>>>>>>>>>>>>>>
>>>>>>>>>>>>> let
>>>>>>
>>>>>>> alone
>>>>>>>>
>>>>>>>>> modify
>>>>>>>>>>>
>>>>>>>>>>>> them to create a testing framework.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Suggested Solution
>>>>>>>>>>>>>>> ------------------------
>>>>>>>>>>>>>>> I suggest to implement the testing framework in Gradle,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>
>>>>>>>>>>>>> simply
>>>>>>>
>>>>>>>> call
>>>>>>>>>>>
>>>>>>>>>>>> it
>>>>>>>>>>>>>
>>>>>>>>>>>>>> from within ant. This is a middle solution that
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> sustains
>>>>>>>>>>>>>>
>>>>>>>>>>>>> ant
>>>>>>
>>>>>>> for
>>>>>>>>
>>>>>>>>> now,
>>>>>>>>>>
>>>>>>>>>>> but
>>>>>>>>>>>>
>>>>>>>>>>>>> can allow us to switch out later.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> This means I will just add one more file called
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> build.gradle
>>>>>>>>>>>>>>
>>>>>>>>>>>>> in
>>>>>>>>
>>>>>>>> the
>>>>>>>>>
>>>>>>>>>> top
>>>>>>>>>>>
>>>>>>>>>>>> level directory, and figure out the business logic for
>>>>>>>>>>>>>
>>>>>>>>>>>>>> calling
>>>>>>>>>>>>>>
>>>>>>>>>>>>> the
>>>>>>>>>
>>>>>>>>>> test
>>>>>>>>>>>
>>>>>>>>>>>> suites from that file
>>>>>>>>>>>>>
>>>>>>>>>>>>>> I look forward to your feedback.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <
>>>>>>>>>>>>>>> [hidden email] <javascript:;>>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Everyone,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thank you all for your support, JIRA created in
>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OFBIZ-7254
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I will start working on it and try to implement ASAP
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> get
>>>>>>
>>>>>>> my
>>>>>>>>
>>>>>>>>> focus
>>>>>>>>>>
>>>>>>>>>>> back
>>>>>>>>>>>>>
>>>>>>>>>>>>>> on refactoring.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Cheers!
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
>>>>>>>>>>>>>>>> [hidden email] <javascript:;>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> +1
>>>>>>
>>>>>>> Thanks & Regards
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> Deepak Dixit
>>>>>>>>>>>>>>>>> www.hotwaxsystems.com
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
>>>>>>>>>>>>>>>>> [hidden email] <javascript:;>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> +1
>>>>>>>
>>>>>>>> Makes perfect sense.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> Thanks & Regards,
>>>>>>>>>>>>>>>>>> Mridul Pathak
>>>>>>>>>>>>>>>>>> Senior Manager
>>>>>>>>>>>>>>>>>> HotWax Systems
>>>>>>>>>>>>>>>>>> http://www.hotwaxsystems.com
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
>>>>>>>>>>>>>>>>>> [hidden email] <javascript:;>>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hello Everyone,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> After refactoring the start component and while
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> starting
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> on
>>>>>>>>>
>>>>>>>>>> the
>>>>>>>>>>>
>>>>>>>>>>>> base
>>>>>>>>>>>>>
>>>>>>>>>>>>>> component I realized that the testing framework
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> OFBiz
>>>>>>>
>>>>>>>> is
>>>>>>>>>>
>>>>>>>>>> not
>>>>>>>>>>>
>>>>>>>>>>>> good.
>>>>>>>>>>>>>
>>>>>>>>>>>>>> You
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> cannot do real test driven development or
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> red-green-refactor
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>
>>>>>>>>>>>> the
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> current setup, hence my proposal to change it. I
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> explain
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> below:
>>>>>>>>>
>>>>>>>>>> Problem with current design
>>>>>>>>>>>>>
>>>>>>>>>>>>>> ----------------------------------------
>>>>>>>>>>>>>>>>>>> - What we have right now is not unit tests, it's
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> really
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> integration
>>>>>>>>>
>>>>>>>>>> tests.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> You have to start the framework, the database,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> service
>>>>>>
>>>>>>> engine,
>>>>>>>>>>
>>>>>>>>>>> the
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> entity engine and pretty much everything.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> - Testing is very slow, because it's an
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> integration
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> test
>>>>>>>
>>>>>>>> as
>>>>>>>>>
>>>>>>>>>> I
>>>>>>>>>>>
>>>>>>>>>>> mentioned
>>>>>>>>>>>>
>>>>>>>>>>>>> above. 10 minutes on a good computer!
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> - There is zero mocking! We actually have to
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --load-data
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> for
>>>>>>>>>
>>>>>>>>>> things
>>>>>>>>>>>
>>>>>>>>>>>> to
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> work. Again, these are integration tests.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> - Too complex: Integration tests by their nature
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> grabbing
>>>>>>>
>>>>>>>> too
>>>>>>>>>>>>
>>>>>>>>>>>>> much.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Mind you, I am not objecting to integration
>>>>>>>>>>>>>>>>>> tests
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> (I
>>>>>>
>>>>>> actually
>>>>>>>
>>>>>>>> like
>>>>>>>>>>>>
>>>>>>>>>>>>> them)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> but I am objecting to not having real
>>>>>>>>>>>>>>>>>> unit-tests.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Unit
>>>>>>
>>>>>>> tests
>>>>>>>>
>>>>>>>>> should
>>>>>>>>>>>
>>>>>>>>>>>> all
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> run
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> in a few seconds.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Proposed solution
>>>>>>>>>>>>>>>>>>> --------------------------
>>>>>>>>>>>>>>>>>>> - We keep what is considered real integration
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> tests
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> the
>>>>>>>
>>>>>>>> way
>>>>>>>>>
>>>>>>>>>> they
>>>>>>>>>>>
>>>>>>>>>>>> are
>>>>>>>>>>>>>
>>>>>>>>>>>>>> right
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> now and keep using them
>>>>>>>>>>>>>>>>>>> - We move what should be unit tests into simple
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> JUnit
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> classes,
>>>>>>>>
>>>>>>>>> and
>>>>>>>>>>>>
>>>>>>>>>>>>> we
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> do
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> not run them using java -jar ofbiz.jar --test,
>>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> instead
>>>>>>
>>>>>>> run
>>>>>>>>>>
>>>>>>>>>>> them
>>>>>>>>>>>>
>>>>>>>>>>>>> directly from the build.xml script, so these
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> files
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> are
>>>>>>
>>>>>>> not
>>>>>>>>
>>>>>>>>> identified
>>>>>>>>>>
>>>>>>>>>>> in
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> any XML document, but are simply called
>>>>>>>>>>>>>>>>>> immediately
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> from
>>>>>>>
>>>>>>>> the
>>>>>>>>>
>>>>>>>>>> build
>>>>>>>>>>>
>>>>>>>>>>>> scripts.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - We clearly mark the difference between
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> integration
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> tests
>>>>>>>
>>>>>>>> and
>>>>>>>>>>
>>>>>>>>>>> unit
>>>>>>>>>>>>
>>>>>>>>>>>>> tests
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> (inside the source files or in the suite
>>>>>>>>>>>>>>>>>> declarations).
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> - We change the run-tests target in build.xml to
>>>>>>>>>
>>>>>>>>>> run
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> both
>>>>>>>
>>>>>>>> unit
>>>>>>>>>>
>>>>>>>>>>> tests
>>>>>>>>>>>>
>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> integration tests.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I intend to heavily refactor the framework and I
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> feel
>>>>>>>>
>>>>>>>>> better
>>>>>>>>>>>
>>>>>>>>>>>> about
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> introducing this change while refactoring. What
>>>>>>>>>>>>>>>>>> do
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> you
>>>>>>
>>>>>>> guys
>>>>>>>>
>>>>>>>>> think?
>>>>>>>>>>>
>>>>>>>>>>>> Ideas?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Suggestions? Approvals and thumbs up?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>
> --
> Ron Wheeler
> President
> Artifact Software Inc
> email: [hidden email]
> skype: ronaldmwheeler
> phone: [phone], ext 102
>
>

[Ron Wheeler]

I think that your point was well received.
I am just happy that testing is getting attention.

Ron

On 19/07/2016 7:53 PM, Scott Gray wrote:

> I'm not looking for any sort of response really.  I was just speaking in
> favor of integration tests because I think they provide better coverage at
> the cost of a little bit of speed (the framework has to be running to run
> them).  Most of the responses coming in were of the form "unit tests are
> great because [insert a reason that applies to any type of test]".  So I
> was merely pointing out that the arguments weren't specific to unit tests
> and hence weren't really adding value to the discussion.
>
> Regards
> Scott
>
> On 20 July 2016 at 02:05, Ron Wheeler <[hidden email]>
> wrote:
>
>> It is not clear what you are expecting as a specific response.
>>
>> As you move up the food chain, the definition of "units" changes and the
>> tests get more complicated and the design of the tests gets more complex
>> (mock objects, testing race conditions).
>>
>> It is hard to make specific comments about what kind of testing will give
>> the highest level of confidence at each level but we are going in the right
>> direction.
>>
>> The fact that we are considering tests as a major development activity is
>> a big step forward.
>>
>> We can argue about specific cases (scope, tools, value, etc) as we get to
>> them.
>>
>> I think that we are at the "good idea" stage rather than at the "policy"
>> stage at this point.
>>
>> Ron
>>
>>
>>
>> On 19/07/2016 4:23 AM, Scott Gray wrote:
>>
>>> I think people are missing my point because they keep replying with
>>> generic
>>> statements that aren't specific to unit tests.
>>>
>>> On 19 July 2016 at 20:08, Montalbano Florian <
>>> [hidden email]>
>>> wrote:
>>>
>>> Hi everyone,
>>>> unit tests and integration tests are complementary and with test, the
>>>> more
>>>> the better.
>>>> Implementing unit tests will be a great load of work but it is the same
>>>> with every kind of test. As Taher said, we will gain a lot from those
>>>> unit
>>>> tests for avoiding regression while refactoring the framework. From the
>>>> start component scope, it has already be proved useful.
>>>> And what Ron said about encouraging the creation of "smaller class that
>>>> can be tested easily" is very important too. It can be a first step to
>>>> reach TDD in the OFBiz development. TDD can help to give confidence to
>>>> newcomer and veteran when updating some part of the code and as Hans
>>>> said,
>>>> this lead to "continuous improvement and finally continuous deployment".
>>>>
>>>> I'm aware that this does not solve the technical implementation stuff,
>>>> but
>>>> I think this is an improvement that need to be supported.
>>>>
>>>> Have a nice day,
>>>>
>>>> Florian Montalbano
>>>>
>>>>
>>>> Le 19/07/2016 09:04, Pierre Smits a écrit :
>>>>
>>>> As with anything, the law of diminishing returns also applies to OFBiz
>>>>> and
>>>>> tests. This is not true for unit tests and system integration tests, but
>>>>> also for user acceptance and performance tests.
>>>>>
>>>>> Nevertheless, the work done up to now is a good start and - I feel
>>>>> confident - appreciated. And unit tests are certainly valuable in the
>>>>> framework stack. How it will be for functions (regarding components in
>>>>> application and special purpose stack) needs to be addressed when we
>>>>> reach
>>>>> that bridge.
>>>>>
>>>>>
>>>>> Best regards,
>>>>>
>>>>> Pierre Smits
>>>>>
>>>>> ORRTIZ.COM <http://www.orrtiz.com>
>>>>> OFBiz based solutions & services
>>>>>
>>>>> OFBiz Extensions Marketplace
>>>>> http://oem.ofbizci.net/oci-2/
>>>>>
>>>>> On Tue, Jul 19, 2016 at 8:22 AM, Taher Alkhateeb <
>>>>> [hidden email]
>>>>>
>>>>> wrote:
>>>>>> Hi Scott,
>>>>>>
>>>>>> Well spoken thank you. Okay may I suggest that for any such work we
>>>>>> will
>>>>>> discuss it here see its Merit and if it makes sense then we take it in.
>>>>>> It
>>>>>> is a little early to discuss it right now because we did not yet go to
>>>>>> the
>>>>>> higher-level components. Once we do I'll be sure to have a conversation
>>>>>> about this in here and would appreciate your input to it.
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> Taher Alkhateeb
>>>>>>
>>>>>> On Jul 19, 2016 9:00 AM, "Scott Gray" <[hidden email]>
>>>>>> wrote:
>>>>>>
>>>>>> Yeah I'm sure unit tests probably worked well for the start component
>>>>>> given
>>>>>>
>>>>>> it is the lowest level component in the system and closest to a basic
>>>>>>> java
>>>>>> app.  I just think the value proposition might decrease the further up
>>>>>>> the
>>>>>> stack we move with them.  I'm not against unit tests when they prove
>>>>>>> useful, but further up the stack I think we should prove the case for
>>>>>>>
>>>>>>> them
>>>>>> before doing much work to support the mocking that will be required to
>>>>>>> keep
>>>>>> them inline with 'best practices'.
>>>>>>> In OFBiz "bad or unwanted" tends to come mostly in the way of
>>>>>>> increasing
>>>>>>> complexity from features that require more effort to maintain than the
>>>>>>> value they provide to the community.  I think there's a chance
>>>>>>> attempting
>>>>>>> to mock service tests could fall into that bucket.  I could be wrong,
>>>>>>>
>>>>>>> but I
>>>>>> think it's worth looking into before we declare that unit tests are the
>>>>>>> best form of testing for OFBiz.
>>>>>>>
>>>>>>> Regards
>>>>>>> Scott
>>>>>>>
>>>>>>> On 19 July 2016 at 17:37, Taher Alkhateeb <[hidden email]
>>>>>>> wrote:
>>>>>>>
>>>>>>> Hi Scott,
>>>>>>>
>>>>>>>> Thank you for the feedback. To be focused exactly on integration vs
>>>>>>>>
>>>>>>>> unit, I
>>>>>>> already mentioned above that at least for me the main objective is to
>>>>>>>> confidently and quickly run the tests in short bursts of red-green
>>>>>>>> refactor. This allows me to refactor code without waiting in front of
>>>>>>>>
>>>>>>>> the
>>>>>>> screen in between test cycles thus giving me immediate feedback on any
>>>>>>>
>>>>>>>> errors I made. Perhaps my intro was too long so this is the squeezed
>>>>>>>>
>>>>>>>> out
>>>>>>> version of it.
>>>>>>>
>>>>>>>> I already had one round of testing in the start component which was
>>>>>>>>
>>>>>>>> much
>>>>>>> faster that way and had an immediate impact. Oh and by the way, you
>>>>>>> cannot
>>>>>>>
>>>>>>> test the start component with integration tests for example unless you
>>>>>>>> do
>>>>>>> it from an external component which cannot access package protected
>>>>>>> items.
>>>>>>>
>>>>>>> This style of coding is applicable I think to any software project
>>>>>>>> inclusive of OFBiz. Maybe in certain components more than others or
>>>>>>>>
>>>>>>>> certain
>>>>>>> areas more than others. But I can't see how it could be bad or
>>>>>>>> unwanted.
>>>>>>> Taher Alkhateeb
>>>>>>>
>>>>>>>> On Jul 19, 2016 8:18 AM, "Scott Gray" <[hidden email]>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Thanks Ron and Taher for your responses, I appreciate them but I
>>>>>>>> don't
>>>>>>>>
>>>>>>> hear
>>>>>>>
>>>>>>>> much in the conversation that speaks to the value of unit tests over
>>>>>>>>> integration tests.  Most of the thoughts shared speaks more to the
>>>>>>>>>
>>>>>>>>> value
>>>>>>>> of
>>>>>>>>
>>>>>>>> tests rather than differences in the type of tests.
>>>>>>>>> Speed: At an application level we have ~685 tests that run in 35
>>>>>>>>>
>>>>>>>>> seconds
>>>>>>>> (excluding build and data load).  Another point is that there isn't
>>>>>>>> much
>>>>>>>> reason why tests can't be run in parallel rather than sequentially.
>>>>>>>>
>>>>>>>>> TDD: Integration tests in OFBiz are as simple if not simpler to
>>>>>>>>> write
>>>>>>>>>
>>>>>>>>> than
>>>>>>>> unit tests and just as useful for TDD.
>>>>>>>>> I'm not sure if I missed any other points raised regarding
>>>>>>>>>
>>>>>>>>> integration
>>>>>>> vs.
>>>>>>>
>>>>>>>> unit tests?
>>>>>>>>> I'm not looking to start a big long debate on the topic, I just
>>>>>>>>>
>>>>>>>>> wanted
>>>>>>> to
>>>>>>>
>>>>>>> speak out that someone out there (me) doesn't think unit tests are
>>>>>>>> the
>>>>>>>>
>>>>>>> best
>>>>>>>
>>>>>>>> solution for testing OFBiz applications.
>>>>>>>>> Regards
>>>>>>>>> Scott
>>>>>>>>>
>>>>>>>>> On 19 July 2016 at 16:52, Taher Alkhateeb <
>>>>>>>>>
>>>>>>>>> [hidden email]>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Scott,
>>>>>>>>>> Thank you for your input. Your ideas are thought provoking and
>>>>>>>>>>
>>>>>>>>>> enriching
>>>>>>>>> to
>>>>>>>>>
>>>>>>>>> the conversation.
>>>>>>>>>> The way I look at it, the general rule is usually many unit tests,
>>>>>>>>>>
>>>>>>>>>> less
>>>>>>>> integration tests, lesser functional tests. So we are not excluding
>>>>>>>>
>>>>>>>>> any
>>>>>>>>>
>>>>>>>> types of test, all of them are important in certain areas for
>>>>>>>>
>>>>>>>>> certain
>>>>>>>>>
>>>>>>>> purposes with certain quantity. Usually integration tests are less
>>>>>>>> because
>>>>>>>>> as you said they just grab more. I like the picture below as a
>>>>>>>>>> general
>>>>>>>> guideline
>>>>>>>>
>>>>>>>>> http://i.stack.imgur.com/fjQvQ.png
>>>>>>>>>> As you already mentioned unit tests are useful for the framework. I
>>>>>>>>>> discovered errors in code that I wrote which I was very very
>>>>>>>>>>
>>>>>>>>>> careful
>>>>>>>> with.
>>>>>>>> I immediately learned the lesson that humans are not designed to
>>>>>>>>>> code,
>>>>>>>> and
>>>>>>>>
>>>>>>>>> TDD gives you confidence as you build your code with those short
>>>>>>>>>> 30-60
>>>>>>>> second red-green refactors. I feel much much safer and more
>>>>>>>>
>>>>>>>>> confident
>>>>>>>>>
>>>>>>>> writing code that way, the test also documents how to use the api,
>>>>>>>> refactoring and feature change becomes less terrifying. Also messy
>>>>>>>>>> code
>>>>>>>> is
>>>>>>>>
>>>>>>>>> usually not test friendly thus refactoring and unit tests go
>>>>>>>>>> hand-in-hand
>>>>>>>>> for improving the code base.
>>>>>>>>>
>>>>>>>>>> Also, I am sorry to say that the framework code is rather messy and
>>>>>>>>>>
>>>>>>>>>> brittle
>>>>>>>>> in many places. I think you probably encountered this yourself. The
>>>>>>>>>> same
>>>>>>>>> is
>>>>>>>>>
>>>>>>>>> said for the applications. Now If we start refactoring without unit
>>>>>>>>>> tests
>>>>>>>>> then we are back to scary business. So much can go wrong so fast
>>>>>>>>> and
>>>>>>>>>
>>>>>>>> break
>>>>>>>> things you never expected to break. The framework with all
>>>>>>>>>> applications
>>>>>>>> require heavy refactoring of things like massive ugly methods,
>>>>>>>>
>>>>>>>>> sandwiched
>>>>>>>>> logic, heavy shared mutable state, hidden dependencies, poor
>>>>>>>>>
>>>>>>>>>> interfaces, and much more. Every time I touch something I get
>>>>>>>>>>
>>>>>>>>>> shocked
>>>>>>>> at
>>>>>>>> how bad it looks, spaghetti logic to no end. If you refactor with
>>>>>>>>>> integration tests instead of unit tests then you will come down to
>>>>>>>>>>
>>>>>>>>>> a
>>>>>>>> screeching halt as you wait between these test cycles. My computer
>>>>>>>> actually
>>>>>>>>> takes 10 minutes or more for a full clean load data and testing.
>>>>>>>>>> Now talking about mocking you raise some interesting points, what
>>>>>>>>>>
>>>>>>>>>> can
>>>>>>>> you
>>>>>>>> mock vs utilizing integration tests? This is an excellent question
>>>>>>>>> that
>>>>>>>>>
>>>>>>>> does not have a simple answer, and you mostly learn as you code,
>>>>>>>>
>>>>>>>>> difficult
>>>>>>>>>
>>>>>>>>> to envision without coding. However simple things like Java
>>>>>>>>>> services
>>>>>>>> can
>>>>>>>> be
>>>>>>>>> mocked with a standard mocking class that you use everywhere.
>>>>>>>>>> Transactions
>>>>>>>>> are difficult to mock and better left to integration tests I think.
>>>>>>>>>> SECAs
>>>>>>>>> might be mocked by simply passing output to input as chaining
>>>>>>>>> methods.
>>>>>>>>>
>>>>>>>> Mind
>>>>>>>>
>>>>>>>>> you I am not 100% sure of all of this, coding is the ultimate guide
>>>>>>>>>> to
>>>>>>>> what
>>>>>>>>
>>>>>>>>> can or cannot be done.
>>>>>>>>>> So I am not suggesting unit tests as a best-practice (even though
>>>>>>>>>>
>>>>>>>>>> it
>>>>>>>> is).
>>>>>>>> Instead I suggest it as something that I and others did and got a
>>>>>>>>> huge
>>>>>>>>>
>>>>>>>> psychological relief and confidence and comfort from. Swimming in
>>>>>>>>
>>>>>>>>> code
>>>>>>>>>
>>>>>>>> without tests is a terrifying business, made more terrifying if the
>>>>>>>>
>>>>>>>>> code
>>>>>>>>> is
>>>>>>>>>
>>>>>>>>> bad. Short bursts of red/green/red/green makes you feel good as you
>>>>>>>>>> build
>>>>>>>>> up your logic. And I don't have an exact vision of how to do it,
>>>>>>>>> because
>>>>>>>>> the details always win.
>>>>>>>>>
>>>>>>>>>> I look forward to hearing your thoughts and thank you for enriching
>>>>>>>>>>
>>>>>>>>>> this
>>>>>>>>> conversation.
>>>>>>>>>
>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>
>>>>>>>>>> On Tuesday, 19 July 2016, Scott Gray <[hidden email]
>>>>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>> I know I'm late to the party here, but I just want to say that I
>>>>>>>>>> think
>>>>>>>>>>
>>>>>>>>> integration tests have far greater value to OFBiz than unit
>>>>>>>>>
>>>>>>>>>> tests.
>>>>>>>>>>
>>>>>>>>> Mostly
>>>>>>>> because we tend to have quite a low number of tests and
>>>>>>>>>>> integration
>>>>>>>>> tests
>>>>>>>> give us much better coverage per line of test code and the tests
>>>>>>>>>> are
>>>>>>>>>>
>>>>>>>>> much
>>>>>>>>> closer to the real world scenarios the application might
>>>>>>>>>> encounter.
>>>>>>>>>>
>>>>>>>>> I don't really see how unit tests could be applied to
>>>>>>>> non-framework
>>>>>>>>> testing
>>>>>>>> in a useful manner, could you expand on your vision in that
>>>>>>>>>>> regard?
>>>>>>>>> I
>>>>>>>> mean
>>>>>>>>>> would we be testing something smaller than a service as the
>>>>>>>>>>> 'unit'?
>>>>>>>>> What
>>>>>>>> would we mock? Would transaction management still be active? What
>>>>>>>>>> happens
>>>>>>>>>> when the service calls another service, I guess we mock the
>>>>>>>>>> response
>>>>>>>>>>
>>>>>>>>> from
>>>>>>>>> that service (how)?
>>>>>>>>>>> It just seems a very complicated method to achieve a less
>>>>>>>>>>>
>>>>>>>>>>> thorough
>>>>>>>>> but
>>>>>>>> albeit faster (maybe) test result.
>>>>>>>>>> A build, data load and full test run takes 4m 9s on my laptop.
>>>>>>>>>>> Excluding
>>>>>>>>>> build, data load and framework tests: the application level tests
>>>>>>>>>> take
>>>>>>>>>>
>>>>>>>>> 35s,
>>>>>>>>>
>>>>>>>>>> not very expensive IMO.  The data load time can be reduced to
>>>>>>>>>>> practically
>>>>>>>>>> nothing by copying a clean slate database into runtime for each
>>>>>>>>>> run.
>>>>>>>>>>
>>>>>>>>> I'm mostly just suggesting we be wary of adding complicated
>>>>>>>>> testing
>>>>>>>>> procedures in the hope of achieving some 'best practice' result
>>>>>>>> which
>>>>>>>>> in
>>>>>>>>> reality will provide minimal benefits.
>>>>>>>>>>> Regards
>>>>>>>>>>> Scott
>>>>>>>>>>>
>>>>>>>>>>> On 18 July 2016 at 18:57, Taher Alkhateeb <
>>>>>>>>>>>
>>>>>>>>>>> [hidden email]
>>>>>>>>> <javascript:;>>
>>>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>> Hello Akash,
>>>>>>>>>>>
>>>>>>>>>>>> Fantastic, I have a few unit tests almost done to be included
>>>>>>>>>>>>
>>>>>>>>>>>> in
>>>>>>>>>> the
>>>>>>>> start
>>>>>>>>>> component. I will create a new subtask under OFBIZ-1463 to
>>>>>>>>>>>> commit
>>>>>>>>>> the
>>>>>>>> tests
>>>>>>>>>> so you can use them as a reference if you like to.
>>>>>>>>>>>> I also recommend that you follow the same directory structure
>>>>>>>>>>>>
>>>>>>>>>>>> between
>>>>>>>>>> the
>>>>>>>>>> test code and production code. So for example:
>>>>>>>>>>>> Production code:
>>>>>>>>>>>>
>>>>>>>>>>>> framework/start/src/main/java/org/apache/ofbiz/base/start
>>>>>>>>>>> Test code:
>>>>>>>>>>>> framework/start/src/test/java/org/apache/ofbiz/base/start
>>>>>>>>>> The benefit of this hierarchy is that you can access non-public
>>>>>>>>>> (package
>>>>>>>>>>> protected) methods for testing. This is in fact exactly what I
>>>>>>>>>>> needed
>>>>>>>>>>>
>>>>>>>>>> to
>>>>>>>>>> be
>>>>>>>>>>> able to apply some of the tests.
>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>
>>>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Jul 18, 2016 at 9:22 AM, Akash Jain <
>>>>>>>>>>>>
>>>>>>>>>>>> [hidden email] <javascript:;>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>> Thanks Taher for nice initiative!
>>>>>>>>>>>>
>>>>>>>>>>>>> We are planning to written unit tests to all components under
>>>>>>>>>>>>>
>>>>>>>>>>>>> OFBIZ-1463
>>>>>>>>>>>> Thanks and Regards
>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Akash Jain
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Jul 18, 2016 at 10:36 AM, Taher Alkhateeb <
>>>>>>>>>>>>> [hidden email] <javascript:;>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hello Everyone,
>>>>>>>>>>>>>
>>>>>>>>>>>>>> In reference to this thread and the Jira OFBIZ-7254, I'm
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> very
>>>>>>>>>>>> happy
>>>>>>>> to
>>>>>>>>>>> announce that OFBiz is now ready for applying unit tests
>>>>>>>>>>>>> with
>>>>>>>>>>>>>
>>>>>>>>>>>> only
>>>>>>>> 8
>>>>>>>>>> new
>>>>>>>>>>>> lines of code in the build script (r1753143) :)
>>>>>>>>>>>>>> I recommend we do the following moving forward:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 1- Introduce unit tests as much as we can to all components
>>>>>>>>>>>>>> 2- Migrate most of the integration tests we currently have
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> to
>>>>>>>>>>>> unit
>>>>>>>> tests
>>>>>>>>>>> (since they are designed to do very little integration).
>>>>>>>>>>>>>> This is a great chance for us to practice real TDD in OFBiz
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Jun 13, 2016 at 7:43 AM, Pranay Pandey <
>>>>>>>>>>>>>> [hidden email] <javascript:;>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> +1
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Best regards,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Pranay Pandey
>>>>>>>>>>>>>>> HotWax Systems
>>>>>>>>>>>>>>> http://www.hotwaxsystems.com/
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Fri, Jun 10, 2016 at 7:46 PM, Taher Alkhateeb <
>>>>>>>>>>>>>>> [hidden email] <javascript:;>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> Hello Everyone,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I was able to get a few tests running and this is very
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> doable.
>>>>>>>>>>>>>> But
>>>>>>>>>>> I
>>>>>>>>>>>> faced
>>>>>>>>>>>>>> a big problem in designing the testing framework
>>>>>>>>>>>>>>>> because
>>>>>>>>>>>>>> of
>>>>>>> ANT.
>>>>>>>>> The problem
>>>>>>>>>>>> ----------------
>>>>>>>>>>>>>>>> The way the build scripts are designed in OFBiz are
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> very
>>>>>>>>>>>>>> complex. A
>>>>>>>> master
>>>>>>>>>>>>> file calls other files which call other files. And in
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>> middle
>>>>>>>> you
>>>>>>>>>>>> have
>>>>>>>>>>>>>> external libraries (ant-contrib) and macros, and
>>>>>>>>>>>>>>> variables,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> and
>>>>>>>>> class
>>>>>>>>>>> path
>>>>>>>>>>>>>> declarations, and and and ....
>>>>>>>>>>>>>>>> I cannot declare the tests programmatically (with JUnit
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> test
>>>>>>>>>>>>>> suites)
>>>>>>>>>> because this means lower level components would depend
>>>>>>>>>>>>>> on
>>>>>>>>>>>>>> higher
>>>>>>>> level
>>>>>>>>>>>> components. So I have to do it in ANT, by navigating
>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> maze
>>>>>>>> of
>>>>>>>>>> build
>>>>>>>>>>>> scripts, and it was a headache for me just to read
>>>>>>>>>>>>>>> them,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> let
>>>>>>>> alone
>>>>>>>>>> modify
>>>>>>>>>>>>> them to create a testing framework.
>>>>>>>>>>>>>>>> Suggested Solution
>>>>>>>>>>>>>>>> ------------------------
>>>>>>>>>>>>>>>> I suggest to implement the testing framework in Gradle,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>> simply
>>>>>>>>> call
>>>>>>>>>>>>> it
>>>>>>>>>>>>>>> from within ant. This is a middle solution that
>>>>>>>>>>>>>>>> sustains
>>>>>>>>>>>>>> ant
>>>>>>>> for
>>>>>>>>>> now,
>>>>>>>>>>>> but
>>>>>>>>>>>>>> can allow us to switch out later.
>>>>>>>>>>>>>>>> This means I will just add one more file called
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> build.gradle
>>>>>>>>>>>>>> in
>>>>>>>>> the
>>>>>>>>>>> top
>>>>>>>>>>>>> level directory, and figure out the business logic for
>>>>>>>>>>>>>>> calling
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> the
>>>>>>>>>>> test
>>>>>>>>>>>>> suites from that file
>>>>>>>>>>>>>>> I look forward to your feedback.
>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <
>>>>>>>>>>>>>>>> [hidden email] <javascript:;>>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi Everyone,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thank you all for your support, JIRA created in
>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OFBIZ-7254
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I will start working on it and try to implement ASAP
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> get
>>>>>>>> my
>>>>>>>>>> focus
>>>>>>>>>>>> back
>>>>>>>>>>>>>>> on refactoring.
>>>>>>>>>>>>>>>>> Cheers!
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
>>>>>>>>>>>>>>>>> [hidden email] <javascript:;>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> +1
>>>>>>>> Thanks & Regards
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> Deepak Dixit
>>>>>>>>>>>>>>>>>> www.hotwaxsystems.com
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
>>>>>>>>>>>>>>>>>> [hidden email] <javascript:;>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> +1
>>>>>>>>> Makes perfect sense.
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> Thanks & Regards,
>>>>>>>>>>>>>>>>>>> Mridul Pathak
>>>>>>>>>>>>>>>>>>> Senior Manager
>>>>>>>>>>>>>>>>>>> HotWax Systems
>>>>>>>>>>>>>>>>>>> http://www.hotwaxsystems.com
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
>>>>>>>>>>>>>>>>>>> [hidden email] <javascript:;>>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hello Everyone,
>>>>>>>>>>>>>>>>>>>> After refactoring the start component and while
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> starting
>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>> the
>>>>>>>>>>>>> base
>>>>>>>>>>>>>>> component I realized that the testing framework
>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>> OFBiz
>>>>>>>>> is
>>>>>>>>>>> not
>>>>>>>>>>>>> good.
>>>>>>>>>>>>>>> You
>>>>>>>>>>>>>>>>>> cannot do real test driven development or
>>>>>>>>>>>>>>>>>>>> red-green-refactor
>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> current setup, hence my proposal to change it. I
>>>>>>>>>>>>>>>>>> explain
>>>>>>>>>>>>>>>>>> below:
>>>>>>>>>>> Problem with current design
>>>>>>>>>>>>>>> ----------------------------------------
>>>>>>>>>>>>>>>>>>>> - What we have right now is not unit tests, it's
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> really
>>>>>>>>>>>>>>>>>> integration
>>>>>>>>>>> tests.
>>>>>>>>>>>>>>>>> You have to start the framework, the database,
>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> service
>>>>>>>> engine,
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> entity engine and pretty much everything.
>>>>>>>>>>>>>>>>>> - Testing is very slow, because it's an
>>>>>>>>>>>>>>>>>>>> integration
>>>>>>>>>>>>>>>>>> test
>>>>>>>>> as
>>>>>>>>>>> I
>>>>>>>>>>>> mentioned
>>>>>>>>>>>>>> above. 10 minutes on a good computer!
>>>>>>>>>>>>>>>>>>>> - There is zero mocking! We actually have to
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> --load-data
>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>> things
>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> work. Again, these are integration tests.
>>>>>>>>>>>>>>>>>> - Too complex: Integration tests by their nature
>>>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>> grabbing
>>>>>>>>> too
>>>>>>>>>>>>>> much.
>>>>>>>>>>>>>>>> Mind you, I am not objecting to integration
>>>>>>>>>>>>>>>>>>> tests
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> (I
>>>>>>> actually
>>>>>>>>> like
>>>>>>>>>>>>>> them)
>>>>>>>>>>>>>>>> but I am objecting to not having real
>>>>>>>>>>>>>>>>>>> unit-tests.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Unit
>>>>>>>> tests
>>>>>>>>>> should
>>>>>>>>>>>>> all
>>>>>>>>>>>>>>>>> run
>>>>>>>>>>>>>>>>>>> in a few seconds.
>>>>>>>>>>>>>>>>>>>> Proposed solution
>>>>>>>>>>>>>>>>>>>> --------------------------
>>>>>>>>>>>>>>>>>>>> - We keep what is considered real integration
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> tests
>>>>>>>>>>>>>>>>>> the
>>>>>>>>> way
>>>>>>>>>>> they
>>>>>>>>>>>>> are
>>>>>>>>>>>>>>> right
>>>>>>>>>>>>>>>>> now and keep using them
>>>>>>>>>>>>>>>>>>>> - We move what should be unit tests into simple
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> JUnit
>>>>>>>>>>>>>>>>>> classes,
>>>>>>>>>> and
>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>> do
>>>>>>>>>>>>>>>>>> not run them using java -jar ofbiz.jar --test,
>>>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> instead
>>>>>>>> run
>>>>>>>>>>>> them
>>>>>>>>>>>>>> directly from the build.xml script, so these
>>>>>>>>>>>>>>>>> files
>>>>>>>>>>>>>>>>>> are
>>>>>>>> not
>>>>>>>>>> identified
>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>> any XML document, but are simply called
>>>>>>>>>>>>>>>>>>> immediately
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> from
>>>>>>>>> the
>>>>>>>>>>> build
>>>>>>>>>>>>> scripts.
>>>>>>>>>>>>>>>> - We clearly mark the difference between
>>>>>>>>>>>>>>>>>>>> integration
>>>>>>>>>>>>>>>>>> tests
>>>>>>>>> and
>>>>>>>>>>>> unit
>>>>>>>>>>>>>> tests
>>>>>>>>>>>>>>>>> (inside the source files or in the suite
>>>>>>>>>>>>>>>>>>> declarations).
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> - We change the run-tests target in build.xml to
>>>>>>>>>>> run
>>>>>>>>>>>>>>>>>> both
>>>>>>>>> unit
>>>>>>>>>>>> tests
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> integration tests.
>>>>>>>>>>>>>>>>>>>> I intend to heavily refactor the framework and I
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>> feel
>>>>>>>>>> better
>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>> introducing this change while refactoring. What
>>>>>>>>>>>>>>>>>>> do
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> you
>>>>>>>> guys
>>>>>>>>>> think?
>>>>>>>>>>>>> Ideas?
>>>>>>>>>>>>>>>> Suggestions? Approvals and thumbs up?
>>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>> --
>> Ron Wheeler
>> President
>> Artifact Software Inc
>> email: [hidden email]
>> skype: ronaldmwheeler
>> phone: [phone], ext 102
>>
>>

--
Ron Wheeler
President
Artifact Software Inc
email: [hidden email]
skype: ronaldmwheeler
phone: [phone], ext 102