Re: svn commit: r1856609 - in /ofbiz/ofbiz-framework/trunk/applications/order: groovyScripts/test/OrderTests.groovy testdef/data/OrderTestData.xml

[Jacques Le Roux]

Hi Suraj,

Since this commit https://ci.apache.org/builders/ofbizTrunkFramework/builds/729 the testAddRequirementTask test fails when done in trunk framework only.

https://ci.apache.org/projects/ofbiz/logs/trunk/framework/html/

See also https://ci.apache.org/builders/ofbizTrunkFramework?numbuilds=40

It works when the plugins are also there: https://ci.apache.org/builders/ofbizTrunkFrameworkPlugins?numbuilds=25

https://ci.apache.org/projects/ofbiz/logs/trunk/plugins/html/

I have no ideas why. Maybe some data issue (only in a plugin? Maybe project, not sure).

|
|

|Thanks|

|Jacques|

|
|

Le 30/03/2019 à 08:12, [hidden email] a écrit :

> Author: surajk
> Date: Sat Mar 30 07:12:25 2019
> New Revision: 1856609
>
> URL: http://svn.apache.org/viewvc?rev=1856609&view=rev
> Log:
> Added: Unit test case for service - CreateReturnAdjustment.
> (OFBIZ-8857)
> Thanks Avnindra Sharma for reporting, Pawan Verma for discussion and Vivek Singh Bisen for providing the updated patch.
>
> Modified:
>      ofbiz/ofbiz-framework/trunk/applications/order/groovyScripts/test/OrderTests.groovy
>      ofbiz/ofbiz-framework/trunk/applications/order/testdef/data/OrderTestData.xml
>
> Modified: ofbiz/ofbiz-framework/trunk/applications/order/groovyScripts/test/OrderTests.groovy
> URL: http://svn.apache.org/viewvc/ofbiz/ofbiz-framework/trunk/applications/order/groovyScripts/test/OrderTests.groovy?rev=1856609&r1=1856608&r2=1856609&view=diff
> ==============================================================================
> --- ofbiz/ofbiz-framework/trunk/applications/order/groovyScripts/test/OrderTests.groovy (original)
> +++ ofbiz/ofbiz-framework/trunk/applications/order/groovyScripts/test/OrderTests.groovy Sat Mar 30 07:12:25 2019
> @@ -1,6 +1,7 @@
>   import org.apache.ofbiz.entity.util.EntityQuery
>   import org.apache.ofbiz.service.ServiceUtil
>   import org.apache.ofbiz.testtools.GroovyScriptTestCase
> +
>   class OrderTests extends GroovyScriptTestCase {
>       void testAddRequirementTask() {
>           Map serviceCtx = [:]
> @@ -10,4 +11,13 @@ class OrderTests extends GroovyScriptTes
>           Map serviceResult = dispatcher.runSync("addRequirementTask", serviceCtx)
>           assert ServiceUtil.isSuccess(serviceResult)
>       }
> -}
> +    void testCreateReturnAdjustment() {
> +        Map serviceCtx = [:]
> +        serviceCtx.amount = '2.0000'
> +        serviceCtx.returnId = '1009'
> +        serviceCtx.userLogin = EntityQuery.use(delegator).from('UserLogin').where('userLoginId', 'system').cache().queryOne()
> +        Map serviceResult = dispatcher.runSync('createReturnAdjustment', serviceCtx)
> +        assert ServiceUtil.isSuccess(serviceResult)
> +        assert serviceResult.returnAdjustmentId != null
> +    }
> +}
> \ No newline at end of file
>
> Modified: ofbiz/ofbiz-framework/trunk/applications/order/testdef/data/OrderTestData.xml
> URL: http://svn.apache.org/viewvc/ofbiz/ofbiz-framework/trunk/applications/order/testdef/data/OrderTestData.xml?rev=1856609&r1=1856608&r2=1856609&view=diff
> ==============================================================================
> --- ofbiz/ofbiz-framework/trunk/applications/order/testdef/data/OrderTestData.xml (original)
> +++ ofbiz/ofbiz-framework/trunk/applications/order/testdef/data/OrderTestData.xml Sat Mar 30 07:12:25 2019
> @@ -54,5 +54,6 @@ under the License.
>       <OrderRole orderId="TEST_DEMO10090" partyId="TestDemoCustomer" roleTypeId="SHIP_TO_CUSTOMER"/>
>       <OrderContactMech orderId="TEST_DEMO10090" contactMechPurposeTypeId="ORDER_EMAIL" contactMechId="TestContactMech"/>
>       <Requirement requirementId="1000" requirementTypeId="CUSTOMER_REQUIREMENT"/>
> +    <ReturnHeader returnId="1009" statusId="RETURN_RECEIVED" returnHeaderTypeId="CUSTOMER_RETURN"/>
>  
>   </entity-engine-xml>
>
>
>

[Jacques Le Roux]

Hi Suraj,

Any chances?

Thanks

Jacques

Le 12/04/2019 à 12:03, Jacques Le Roux a écrit :

> Hi Suraj,
>
> Since this commit https://ci.apache.org/builders/ofbizTrunkFramework/builds/729 the testAddRequirementTask test fails when done in trunk framework
> only.
>
> https://ci.apache.org/projects/ofbiz/logs/trunk/framework/html/
>
> See also https://ci.apache.org/builders/ofbizTrunkFramework?numbuilds=40
>
> It works when the plugins are also there: https://ci.apache.org/builders/ofbizTrunkFrameworkPlugins?numbuilds=25
>
> https://ci.apache.org/projects/ofbiz/logs/trunk/plugins/html/
>
> I have no ideas why. Maybe some data issue (only in a plugin? Maybe project, not sure).
>
> |
> |
>
> |Thanks|
>
> |Jacques|
>
> |
> |
>
> Le 30/03/2019 à 08:12, [hidden email] a écrit :
>> Author: surajk
>> Date: Sat Mar 30 07:12:25 2019
>> New Revision: 1856609
>>
>> URL: http://svn.apache.org/viewvc?rev=1856609&view=rev
>> Log:
>> Added: Unit test case for service - CreateReturnAdjustment.
>> (OFBIZ-8857)
>> Thanks Avnindra Sharma for reporting, Pawan Verma for discussion and Vivek Singh Bisen for providing the updated patch.
>>
>> Modified:
>> ofbiz/ofbiz-framework/trunk/applications/order/groovyScripts/test/OrderTests.groovy
>> ofbiz/ofbiz-framework/trunk/applications/order/testdef/data/OrderTestData.xml
>>
>> Modified: ofbiz/ofbiz-framework/trunk/applications/order/groovyScripts/test/OrderTests.groovy
>> URL:
>> http://svn.apache.org/viewvc/ofbiz/ofbiz-framework/trunk/applications/order/groovyScripts/test/OrderTests.groovy?rev=1856609&r1=1856608&r2=1856609&view=diff
>> ==============================================================================
>> --- ofbiz/ofbiz-framework/trunk/applications/order/groovyScripts/test/OrderTests.groovy (original)
>> +++ ofbiz/ofbiz-framework/trunk/applications/order/groovyScripts/test/OrderTests.groovy Sat Mar 30 07:12:25 2019
>> @@ -1,6 +1,7 @@
>>   import org.apache.ofbiz.entity.util.EntityQuery
>>   import org.apache.ofbiz.service.ServiceUtil
>>   import org.apache.ofbiz.testtools.GroovyScriptTestCase
>> +
>>   class OrderTests extends GroovyScriptTestCase {
>>       void testAddRequirementTask() {
>>           Map serviceCtx = [:]
>> @@ -10,4 +11,13 @@ class OrderTests extends GroovyScriptTes
>>           Map serviceResult = dispatcher.runSync("addRequirementTask", serviceCtx)
>>           assert ServiceUtil.isSuccess(serviceResult)
>>       }
>> -}
>> +    void testCreateReturnAdjustment() {
>> +        Map serviceCtx = [:]
>> +        serviceCtx.amount = '2.0000'
>> +        serviceCtx.returnId = '1009'
>> +        serviceCtx.userLogin = EntityQuery.use(delegator).from('UserLogin').where('userLoginId', 'system').cache().queryOne()
>> +        Map serviceResult = dispatcher.runSync('createReturnAdjustment', serviceCtx)
>> +        assert ServiceUtil.isSuccess(serviceResult)
>> +        assert serviceResult.returnAdjustmentId != null
>> +    }
>> +}
>> \ No newline at end of file
>>
>> Modified: ofbiz/ofbiz-framework/trunk/applications/order/testdef/data/OrderTestData.xml
>> URL:
>> http://svn.apache.org/viewvc/ofbiz/ofbiz-framework/trunk/applications/order/testdef/data/OrderTestData.xml?rev=1856609&r1=1856608&r2=1856609&view=diff
>> ==============================================================================
>> --- ofbiz/ofbiz-framework/trunk/applications/order/testdef/data/OrderTestData.xml (original)
>> +++ ofbiz/ofbiz-framework/trunk/applications/order/testdef/data/OrderTestData.xml Sat Mar 30 07:12:25 2019
>> @@ -54,5 +54,6 @@ under the License.
>>       <OrderRole orderId="TEST_DEMO10090" partyId="TestDemoCustomer" roleTypeId="SHIP_TO_CUSTOMER"/>
>>       <OrderContactMech orderId="TEST_DEMO10090" contactMechPurposeTypeId="ORDER_EMAIL" contactMechId="TestContactMech"/>
>>       <Requirement requirementId="1000" requirementTypeId="CUSTOMER_REQUIREMENT"/>
>> +    <ReturnHeader returnId="1009" statusId="RETURN_RECEIVED" returnHeaderTypeId="CUSTOMER_RETURN"/>
>>     </entity-engine-xml>
>>
>>
>>
>

[Suraj Khurana-2]

Hello Jacques,

Thanks for reporting this.

Interestingly, I checked and got fail in a case when I have plugins as well
in the directory structure.

I am using this command to check single test:
./gradlew "ofbiz --test component=order --test suitename=ordertests --test
case=order-tests"

It works fine when I run ./gradlew testIntegration. On a cursory view, IMO,
data is also not dependent anywhere in the plugins directory, so this
should not be the case.

I will have a deeper look into it and provide more details soon (probably
during the weekend).

--
Best Regards,
Suraj Khurana
TECHNICAL CONSULTANT
mobile: [phone]
email: [hidden email]
www.hotwax.co






On Tue, Apr 16, 2019 at 3:01 PM Jacques Le Roux <
[hidden email]> wrote:

> Hi Suraj,
>
> Any chances?
>
> Thanks
>
> Jacques
>
> Le 12/04/2019 à 12:03, Jacques Le Roux a écrit :
> > Hi Suraj,
> >
> > Since this commit
> https://ci.apache.org/builders/ofbizTrunkFramework/builds/729 the
> testAddRequirementTask test fails when done in trunk framework
> > only.
> >
> > https://ci.apache.org/projects/ofbiz/logs/trunk/framework/html/
> >
> > See also https://ci.apache.org/builders/ofbizTrunkFramework?numbuilds=40
> >
> > It works when the plugins are also there:
> https://ci.apache.org/builders/ofbizTrunkFrameworkPlugins?numbuilds=25
> >
> > https://ci.apache.org/projects/ofbiz/logs/trunk/plugins/html/
> >
> > I have no ideas why. Maybe some data issue (only in a plugin? Maybe
> project, not sure).
> >
> > |
> > |
> >
> > |Thanks|
> >
> > |Jacques|
> >
> > |
> > |
> >
> > Le 30/03/2019 à 08:12, [hidden email] a écrit :
> >> Author: surajk
> >> Date: Sat Mar 30 07:12:25 2019
> >> New Revision: 1856609
> >>
> >> URL: http://svn.apache.org/viewvc?rev=1856609&view=rev
> >> Log:
> >> Added: Unit test case for service - CreateReturnAdjustment.
> >> (OFBIZ-8857)
> >> Thanks Avnindra Sharma for reporting, Pawan Verma for discussion and
> Vivek Singh Bisen for providing the updated patch.
> >>
> >> Modified:
> >>
> ofbiz/ofbiz-framework/trunk/applications/order/groovyScripts/test/OrderTests.groovy
> >>
> ofbiz/ofbiz-framework/trunk/applications/order/testdef/data/OrderTestData.xml
> >>
> >> Modified:
> ofbiz/ofbiz-framework/trunk/applications/order/groovyScripts/test/OrderTests.groovy
> >> URL:
> >>
> http://svn.apache.org/viewvc/ofbiz/ofbiz-framework/trunk/applications/order/groovyScripts/test/OrderTests.groovy?rev=1856609&r1=1856608&r2=1856609&view=diff
> >>
> ==============================================================================
> >> ---
> ofbiz/ofbiz-framework/trunk/applications/order/groovyScripts/test/OrderTests.groovy
> (original)
> >> +++
> ofbiz/ofbiz-framework/trunk/applications/order/groovyScripts/test/OrderTests.groovy
> Sat Mar 30 07:12:25 2019
> >> @@ -1,6 +1,7 @@
> >>   import org.apache.ofbiz.entity.util.EntityQuery
> >>   import org.apache.ofbiz.service.ServiceUtil
> >>   import org.apache.ofbiz.testtools.GroovyScriptTestCase
> >> +
> >>   class OrderTests extends GroovyScriptTestCase {
> >>       void testAddRequirementTask() {
> >>           Map serviceCtx = [:]
> >> @@ -10,4 +11,13 @@ class OrderTests extends GroovyScriptTes
> >>           Map serviceResult = dispatcher.runSync("addRequirementTask",
> serviceCtx)
> >>           assert ServiceUtil.isSuccess(serviceResult)
> >>       }
> >> -}
> >> +    void testCreateReturnAdjustment() {
> >> +        Map serviceCtx = [:]
> >> +        serviceCtx.amount = '2.0000'
> >> +        serviceCtx.returnId = '1009'
> >> +        serviceCtx.userLogin =
> EntityQuery.use(delegator).from('UserLogin').where('userLoginId',
> 'system').cache().queryOne()
> >> +        Map serviceResult =
> dispatcher.runSync('createReturnAdjustment', serviceCtx)
> >> +        assert ServiceUtil.isSuccess(serviceResult)
> >> +        assert serviceResult.returnAdjustmentId != null
> >> +    }
> >> +}
> >> \ No newline at end of file
> >>
> >> Modified:
> ofbiz/ofbiz-framework/trunk/applications/order/testdef/data/OrderTestData.xml
> >> URL:
> >>
> http://svn.apache.org/viewvc/ofbiz/ofbiz-framework/trunk/applications/order/testdef/data/OrderTestData.xml?rev=1856609&r1=1856608&r2=1856609&view=diff
> >>
> ==============================================================================
> >> ---
> ofbiz/ofbiz-framework/trunk/applications/order/testdef/data/OrderTestData.xml
> (original)
> >> +++
> ofbiz/ofbiz-framework/trunk/applications/order/testdef/data/OrderTestData.xml
> Sat Mar 30 07:12:25 2019
> >> @@ -54,5 +54,6 @@ under the License.
> >>       <OrderRole orderId="TEST_DEMO10090" partyId="TestDemoCustomer"
> roleTypeId="SHIP_TO_CUSTOMER"/>
> >>       <OrderContactMech orderId="TEST_DEMO10090"
> contactMechPurposeTypeId="ORDER_EMAIL" contactMechId="TestContactMech"/>
> >>       <Requirement requirementId="1000"
> requirementTypeId="CUSTOMER_REQUIREMENT"/>
> >> +    <ReturnHeader returnId="1009" statusId="RETURN_RECEIVED"
> returnHeaderTypeId="CUSTOMER_RETURN"/>
> >>     </entity-engine-xml>
> >>
> >>
> >>
> >
>

[Jacques Le Roux]

Thanks Suraj,

Jacques

Le 16/04/2019 à 12:33, Suraj Khurana a écrit :

> Hello Jacques,
>
> Thanks for reporting this.
>
> Interestingly, I checked and got fail in a case when I have plugins as well
> in the directory structure.
>
> I am using this command to check single test:
> ./gradlew "ofbiz --test component=order --test suitename=ordertests --test
> case=order-tests"
>
> It works fine when I run ./gradlew testIntegration. On a cursory view, IMO,
> data is also not dependent anywhere in the plugins directory, so this
> should not be the case.
>
> I will have a deeper look into it and provide more details soon (probably
> during the weekend).
>
> --
> Best Regards,
> Suraj Khurana
> TECHNICAL CONSULTANT
> mobile: [phone]
> email: [hidden email]
> www.hotwax.co
>
>
>
>
>
>
> On Tue, Apr 16, 2019 at 3:01 PM Jacques Le Roux <
> [hidden email]> wrote:
>
>> Hi Suraj,
>>
>> Any chances?
>>
>> Thanks
>>
>> Jacques
>>
>> Le 12/04/2019 à 12:03, Jacques Le Roux a écrit :
>>> Hi Suraj,
>>>
>>> Since this commit
>> https://ci.apache.org/builders/ofbizTrunkFramework/builds/729 the
>> testAddRequirementTask test fails when done in trunk framework
>>> only.
>>>
>>> https://ci.apache.org/projects/ofbiz/logs/trunk/framework/html/
>>>
>>> See also https://ci.apache.org/builders/ofbizTrunkFramework?numbuilds=40
>>>
>>> It works when the plugins are also there:
>> https://ci.apache.org/builders/ofbizTrunkFrameworkPlugins?numbuilds=25
>>> https://ci.apache.org/projects/ofbiz/logs/trunk/plugins/html/
>>>
>>> I have no ideas why. Maybe some data issue (only in a plugin? Maybe
>> project, not sure).
>>> |
>>> |
>>>
>>> |Thanks|
>>>
>>> |Jacques|
>>>
>>> |
>>> |
>>>
>>> Le 30/03/2019 à 08:12, [hidden email] a écrit :
>>>> Author: surajk
>>>> Date: Sat Mar 30 07:12:25 2019
>>>> New Revision: 1856609
>>>>
>>>> URL: http://svn.apache.org/viewvc?rev=1856609&view=rev
>>>> Log:
>>>> Added: Unit test case for service - CreateReturnAdjustment.
>>>> (OFBIZ-8857)
>>>> Thanks Avnindra Sharma for reporting, Pawan Verma for discussion and
>> Vivek Singh Bisen for providing the updated patch.
>>>> Modified:
>>>>
>> ofbiz/ofbiz-framework/trunk/applications/order/groovyScripts/test/OrderTests.groovy
>> ofbiz/ofbiz-framework/trunk/applications/order/testdef/data/OrderTestData.xml
>>>> Modified:
>> ofbiz/ofbiz-framework/trunk/applications/order/groovyScripts/test/OrderTests.groovy
>>>> URL:
>>>>
>> http://svn.apache.org/viewvc/ofbiz/ofbiz-framework/trunk/applications/order/groovyScripts/test/OrderTests.groovy?rev=1856609&r1=1856608&r2=1856609&view=diff
>> ==============================================================================
>>>> ---
>> ofbiz/ofbiz-framework/trunk/applications/order/groovyScripts/test/OrderTests.groovy
>> (original)
>>>> +++
>> ofbiz/ofbiz-framework/trunk/applications/order/groovyScripts/test/OrderTests.groovy
>> Sat Mar 30 07:12:25 2019
>>>> @@ -1,6 +1,7 @@
>>>>    import org.apache.ofbiz.entity.util.EntityQuery
>>>>    import org.apache.ofbiz.service.ServiceUtil
>>>>    import org.apache.ofbiz.testtools.GroovyScriptTestCase
>>>> +
>>>>    class OrderTests extends GroovyScriptTestCase {
>>>>        void testAddRequirementTask() {
>>>>            Map serviceCtx = [:]
>>>> @@ -10,4 +11,13 @@ class OrderTests extends GroovyScriptTes
>>>>            Map serviceResult = dispatcher.runSync("addRequirementTask",
>> serviceCtx)
>>>>            assert ServiceUtil.isSuccess(serviceResult)
>>>>        }
>>>> -}
>>>> +    void testCreateReturnAdjustment() {
>>>> +        Map serviceCtx = [:]
>>>> +        serviceCtx.amount = '2.0000'
>>>> +        serviceCtx.returnId = '1009'
>>>> +        serviceCtx.userLogin =
>> EntityQuery.use(delegator).from('UserLogin').where('userLoginId',
>> 'system').cache().queryOne()
>>>> +        Map serviceResult =
>> dispatcher.runSync('createReturnAdjustment', serviceCtx)
>>>> +        assert ServiceUtil.isSuccess(serviceResult)
>>>> +        assert serviceResult.returnAdjustmentId != null
>>>> +    }
>>>> +}
>>>> \ No newline at end of file
>>>>
>>>> Modified:
>> ofbiz/ofbiz-framework/trunk/applications/order/testdef/data/OrderTestData.xml
>>>> URL:
>>>>
>> http://svn.apache.org/viewvc/ofbiz/ofbiz-framework/trunk/applications/order/testdef/data/OrderTestData.xml?rev=1856609&r1=1856608&r2=1856609&view=diff
>> ==============================================================================
>>>> ---
>> ofbiz/ofbiz-framework/trunk/applications/order/testdef/data/OrderTestData.xml
>> (original)
>>>> +++
>> ofbiz/ofbiz-framework/trunk/applications/order/testdef/data/OrderTestData.xml
>> Sat Mar 30 07:12:25 2019
>>>> @@ -54,5 +54,6 @@ under the License.
>>>>        <OrderRole orderId="TEST_DEMO10090" partyId="TestDemoCustomer"
>> roleTypeId="SHIP_TO_CUSTOMER"/>
>>>>        <OrderContactMech orderId="TEST_DEMO10090"
>> contactMechPurposeTypeId="ORDER_EMAIL" contactMechId="TestContactMech"/>
>>>>        <Requirement requirementId="1000"
>> requirementTypeId="CUSTOMER_REQUIREMENT"/>
>>>> +    <ReturnHeader returnId="1009" statusId="RETURN_RECEIVED"
>> returnHeaderTypeId="CUSTOMER_RETURN"/>
>>>>      </entity-engine-xml>
>>>>
>>>>
>>>>

[Mathieu Lirzin]

Hello Suraj,

Suraj Khurana <[hidden email]> writes:

> Interestingly, I checked and got fail in a case when I have plugins as well
> in the directory structure.
>
> I am using this command to check single test:
> ./gradlew "ofbiz --test component=order --test suitename=ordertests --test
> case=order-tests"

It will not work because the “order-tests” test case implicitly depends
on the “order-tests-data-load” (yes it sucks...).  So you need to run
the whole test suite:

  ./gradlew "ofbiz --test component=order --test suitename=ordertests"

> It works fine when I run ./gradlew testIntegration. On a cursory view, IMO,
> data is also not dependent anywhere in the plugins directory, so this
> should not be the case.
>
> I will have a deeper look into it and provide more details soon (probably
> during the weekend).

I have been looking at this issue this evening, and after some hours I
still do not understand the bug.  It is depressing to see how hard it is
to work with OFBiz integration tests.

If you don't find a solution before the end of the week, please revert
revision 1856596 which introduces the issue (meaning removing
“OrderTests.groovy”).

Thanks.

--
Mathieu Lirzin
GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37

[Pierre Smits-3]

Removing the code element that is causing the CI to fail is not the right
solution. It will mask the issue (and is like saying 'there is nothing
wrong with the code'), only to reappear in next iteration (when similar is
introduced).

Loading of test data should be happening on suite level, before any
test-case is executed.

Currently the code block in [1] is like:

<test-suite suite-name="suite-name" ..>

<test-case case-name="load-test data">

<entity-xml action="load"
entity-xml-url="component://<component-name/>testdef/data/<name>TestData.xml"/>

</test-case>

<test-case case-name="test-to-execute">

<simple-method-test
location="component://<componen-name>/<path-name>/<test-name>Test.xml"/>

</test-case>

</test-suite>


The example above treats the loading of the test data as a test-case. Which
is incorrect. Loading test data for a component's (suite of) test(s)  is
not about testing the load process, but about prepping the environment for
one or more executions of test-cases of the component.

Taking the load process to the suite level would solve this. Like in
example below:

<test-suite suite-name="suite-name" ..>

<!-- prep for test-cases -->

<entity-xml action="load"
entity-xml-url="component://<component-name/>testdef/data/<name>TestData.xml"/>

<!-- test cases -->

<test-case case-name="test-to-execute">

<simple-method-test
location="component://<componen-name>/<path-name>/<test-name>Test.xml"/>

</test-case>

</test-suite>



Best regards,

Pierre Smits

*Apache Trafodion <https://trafodion.apache.org>, Vice President*
*Apache Directory <https://directory.apache.org>, PMC Member*
Apache Incubator <https://incubator.apache.org>, committer
*Apache OFBiz <https://ofbiz.apache.org>, contributor (without privileges)
since 2008*
Apache Steve <https://steve.apache.org>, committer


On Sat, Apr 20, 2019 at 2:20 AM Mathieu Lirzin <[hidden email]>
wrote:

> Hello Suraj,
>
> Suraj Khurana <[hidden email]> writes:
>
> > Interestingly, I checked and got fail in a case when I have plugins as
> well
> > in the directory structure.
> >
> > I am using this command to check single test:
> > ./gradlew "ofbiz --test component=order --test suitename=ordertests
> --test
> > case=order-tests"
>
> It will not work because the “order-tests” test case implicitly depends
> on the “order-tests-data-load” (yes it sucks...).  So you need to run
> the whole test suite:
>
>   ./gradlew "ofbiz --test component=order --test suitename=ordertests"
>
> > It works fine when I run ./gradlew testIntegration. On a cursory view,
> IMO,
> > data is also not dependent anywhere in the plugins directory, so this
> > should not be the case.
> >
> > I will have a deeper look into it and provide more details soon (probably
> > during the weekend).
>
> I have been looking at this issue this evening, and after some hours I
> still do not understand the bug.  It is depressing to see how hard it is
> to work with OFBiz integration tests.
>
> If you don't find a solution before the end of the week, please revert
> revision 1856596 which introduces the issue (meaning removing
> “OrderTests.groovy”).
>
> Thanks.
>
> --
> Mathieu Lirzin
> GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37
>

[Mathieu Lirzin]

Hello Pierre,

Pierre Smits <[hidden email]> writes:

> Loading of test data should be happening on suite level, before any
> test-case is executed.
>
> Currently the code block in [1] is like:
>
> <test-suite suite-name="suite-name" ..>
>
> <test-case case-name="load-test data">
>
> <entity-xml action="load"
> entity-xml-url="component://<component-name/>testdef/data/<name>TestData.xml"/>
>
> </test-case>
>
> <test-case case-name="test-to-execute">
>
> <simple-method-test
> location="component://<componen-name>/<path-name>/<test-name>Test.xml"/>
>
> </test-case>
>
> </test-suite>
>
>
> The example above treats the loading of the test data as a test-case. Which
> is incorrect. Loading test data for a component's (suite of) test(s)  is
> not about testing the load process, but about prepping the environment for
> one or more executions of test-cases of the component.
>
> Taking the load process to the suite level would solve this. Like in
> example below:
>
> <test-suite suite-name="suite-name" ..>
>
> <!-- prep for test-cases -->
>
> <entity-xml action="load"
> entity-xml-url="component://<component-name/>testdef/data/<name>TestData.xml"/>
>
> <!-- test cases -->
>
> <test-case case-name="test-to-execute">
>
> <simple-method-test
> location="component://<componen-name>/<path-name>/<test-name>Test.xml"/>
>
> </test-case>
>
> </test-suite>

Moving the <entity-xml> element in the children of the <test-suite> one
has currently no effect.  Please make it clear when you are speaking
about not implemented features, I just loose half an hour because of
that.

Currently only the following syntax is “working”:

--8<---------------cut here---------------start------------->8---
<test-case>
  <entity-xml action="load" entity-xml-url="..."/>
</test-case>
--8<---------------cut here---------------end--------------->8---

--
Mathieu Lirzin
GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37

[Pierre Smits-3]

HI Mathieu,

My apologies for the inconvenience caused. I should have made it more clear
that the suggestion was just that. And that the 'would' in the following
sentence should have been 'could':

Taking the load process to the suite level would solve this



Best regards,

Pierre Smits

*Apache Trafodion <https://trafodion.apache.org>, Vice President*
*Apache Directory <https://directory.apache.org>, PMC Member*
Apache Incubator <https://incubator.apache.org>, committer
*Apache OFBiz <https://ofbiz.apache.org>, contributor (without privileges)
since 2008*
Apache Steve <https://steve.apache.org>, committer


On Sat, Apr 20, 2019 at 12:29 PM Mathieu Lirzin <[hidden email]>
wrote:

> Hello Pierre,
>
> Pierre Smits <[hidden email]> writes:
>
> > Loading of test data should be happening on suite level, before any
> > test-case is executed.
> >
> > Currently the code block in [1] is like:
> >
> > <test-suite suite-name="suite-name" ..>
> >
> > <test-case case-name="load-test data">
> >
> > <entity-xml action="load"
> >
> entity-xml-url="component://<component-name/>testdef/data/<name>TestData.xml"/>
> >
> > </test-case>
> >
> > <test-case case-name="test-to-execute">
> >
> > <simple-method-test
> > location="component://<componen-name>/<path-name>/<test-name>Test.xml"/>
> >
> > </test-case>
> >
> > </test-suite>
> >
> >
> > The example above treats the loading of the test data as a test-case.
> Which
> > is incorrect. Loading test data for a component's (suite of) test(s)  is
> > not about testing the load process, but about prepping the environment
> for
> > one or more executions of test-cases of the component.
> >
> > Taking the load process to the suite level would solve this. Like in
> > example below:
> >
> > <test-suite suite-name="suite-name" ..>
> >
> > <!-- prep for test-cases -->
> >
> > <entity-xml action="load"
> >
> entity-xml-url="component://<component-name/>testdef/data/<name>TestData.xml"/>
> >
> > <!-- test cases -->
> >
> > <test-case case-name="test-to-execute">
> >
> > <simple-method-test
> > location="component://<componen-name>/<path-name>/<test-name>Test.xml"/>
> >
> > </test-case>
> >
> > </test-suite>
>
> Moving the <entity-xml> element in the children of the <test-suite> one
> has currently no effect.  Please make it clear when you are speaking
> about not implemented features, I just loose half an hour because of
> that.
>
> Currently only the following syntax is “working”:
>
> --8<---------------cut here---------------start------------->8---
> <test-case>
>   <entity-xml action="load" entity-xml-url="..."/>
> </test-case>
> --8<---------------cut here---------------end--------------->8---
>
> --
> Mathieu Lirzin
> GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37
>

[Pierre Smits-3]

I believe there are a few more where testing individual test-suites and/or
test-cases are dependent on data loaded in other test-suites and/or other
test-cases.

While I don't hear/read about failing testIntegration (except where code in
the base is faulty, not when test-suites/cases are faulty), I see following
failures in test executions in OFBiz against jdk11:


   1. Execution failed for task ':ofbiz --test component=webapp --test
   suitename=webapptests'.
   2. Execution failed for task ':ofbiz --test component=accounting --test
   suitename=invoicetest'.
   3. Execution failed for task ':ofbiz --test component=order --test
   suitename=ordertests'.
   4. Execution failed for task ':ofbiz --test component=product --test
   suitename=producttests'.

Do we have these test failing also when doing the test execution against
jdk8?
*Caveat: I recently set this up, so there may still be some configuration
issues in the jdk11-test setup.. *

Best regards,

Pierre Smits

*Apache Trafodion <https://trafodion.apache.org>, Vice President*
*Apache Directory <https://directory.apache.org>, PMC Member*
Apache Incubator <https://incubator.apache.org>, committer
*Apache OFBiz <https://ofbiz.apache.org>, contributor (without privileges)
since 2008*
Apache Steve <https://steve.apache.org>, committer


On Sat, Apr 20, 2019 at 11:40 AM Pierre Smits <[hidden email]>
wrote:

> Removing the code element that is causing the CI to fail is not the right
> solution. It will mask the issue (and is like saying 'there is nothing
> wrong with the code'), only to reappear in next iteration (when similar is
> introduced).
>
> Loading of test data should be happening on suite level, before any
> test-case is executed.
>
> Currently the code block in [1] is like:
>
> <test-suite suite-name="suite-name" ..>
>
> <test-case case-name="load-test data">
>
> <entity-xml action="load"
> entity-xml-url="component://<component-name/>testdef/data/<name>TestData.xml"/>
>
> </test-case>
>
> <test-case case-name="test-to-execute">
>
> <simple-method-test
> location="component://<componen-name>/<path-name>/<test-name>Test.xml"/>
>
> </test-case>
>
> </test-suite>
>
>
> The example above treats the loading of the test data as a test-case.
> Which is incorrect. Loading test data for a component's (suite of) test(s)
>  is not about testing the load process, but about prepping the environment
> for one or more executions of test-cases of the component.
>
> Taking the load process to the suite level would solve this. Like in
> example below:
>
> <test-suite suite-name="suite-name" ..>
>
> <!-- prep for test-cases -->
>
> <entity-xml action="load"
> entity-xml-url="component://<component-name/>testdef/data/<name>TestData.xml"/>
>
> <!-- test cases -->
>
> <test-case case-name="test-to-execute">
>
> <simple-method-test
> location="component://<componen-name>/<path-name>/<test-name>Test.xml"/>
>
> </test-case>
>
> </test-suite>
>
>
>
> Best regards,
>
> Pierre Smits
>
> *Apache Trafodion <https://trafodion.apache.org>, Vice President*
> *Apache Directory <https://directory.apache.org>, PMC Member*
> Apache Incubator <https://incubator.apache.org>, committer
> *Apache OFBiz <https://ofbiz.apache.org>, contributor (without privileges)
> since 2008*
> Apache Steve <https://steve.apache.org>, committer
>
>
> On Sat, Apr 20, 2019 at 2:20 AM Mathieu Lirzin <[hidden email]>
> wrote:
>
>> Hello Suraj,
>>
>> Suraj Khurana <[hidden email]> writes:
>>
>> > Interestingly, I checked and got fail in a case when I have plugins as
>> well
>> > in the directory structure.
>> >
>> > I am using this command to check single test:
>> > ./gradlew "ofbiz --test component=order --test suitename=ordertests
>> --test
>> > case=order-tests"
>>
>> It will not work because the “order-tests” test case implicitly depends
>> on the “order-tests-data-load” (yes it sucks...).  So you need to run
>> the whole test suite:
>>
>>   ./gradlew "ofbiz --test component=order --test suitename=ordertests"
>>
>> > It works fine when I run ./gradlew testIntegration. On a cursory view,
>> IMO,
>> > data is also not dependent anywhere in the plugins directory, so this
>> > should not be the case.
>> >
>> > I will have a deeper look into it and provide more details soon
>> (probably
>> > during the weekend).
>>
>> I have been looking at this issue this evening, and after some hours I
>> still do not understand the bug.  It is depressing to see how hard it is
>> to work with OFBiz integration tests.
>>
>> If you don't find a solution before the end of the week, please revert
>> revision 1856596 which introduces the issue (meaning removing
>> “OrderTests.groovy”).
>>
>> Thanks.
>>
>> --
>> Mathieu Lirzin
>> GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37
>>
>

[Mathieu Lirzin]

Pierre Smits <[hidden email]> writes:

> I believe there are a few more where testing individual test-suites and/or
> test-cases are dependent on data loaded in other test-suites and/or other
> test-cases.

I have the same experience.  Moreover another source of fragility is
that tests depend on other tests within a single OFBiz “test-case”,
meaning one test can depend on the data produced by another test.  This
is acceptable for a “simple-method-test” because the order of execution
is sequential and managed by OFBiz, but this is problematic for JUnit
tests (Groovy, Java) because the order while being deterministic depends
on the arbitrary order imposed by the JVM.

For example I know for a fact that “QuoteTests.groovy” is suffering from
that issue.

> While I don't hear/read about failing testIntegration (except where code in
> the base is faulty, not when test-suites/cases are faulty), I see following
> failures in test executions in OFBiz against jdk11:
>
>
>    1. Execution failed for task ':ofbiz --test component=webapp --test
>    suitename=webapptests'.
>    2. Execution failed for task ':ofbiz --test component=accounting --test
>    suitename=invoicetest'.
>    3. Execution failed for task ':ofbiz --test component=order --test
>    suitename=ordertests'.
>    4. Execution failed for task ':ofbiz --test component=product --test
>    suitename=producttests'.
>
> Do we have these test failing also when doing the test execution against
> jdk8?
> *Caveat: I recently set this up, so there may still be some configuration
> issues in the jdk11-test setup.. *

I have just tested the “ordertests” test-suite with Icedtea 3.7 (jdk-8)
and it is still failing, so it seems unrelated in that case.

Thanks.

--
Mathieu Lirzin
GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37

[Pierre Smits-3]

Maybe we should move the load aspects regarding tests out of the test suite
invocations altogether.
The gradlew tasks states:

task testIntegration(group: ofbizServer) {

dependsOn 'ofbiz --test'

description 'Run OFBiz integration tests; You must run loadAll before
running this task'

}


IMO, loading test data could be part of the loadAll task.


Best regards,

Pierre Smits

*Apache Trafodion <https://trafodion.apache.org>, Vice President*
*Apache Directory <https://directory.apache.org>, PMC Member*
Apache Incubator <https://incubator.apache.org>, committer
*Apache OFBiz <https://ofbiz.apache.org>, contributor (without privileges)
since 2008*
Apache Steve <https://steve.apache.org>, committer


On Sat, Apr 20, 2019 at 1:56 PM Mathieu Lirzin <[hidden email]>
wrote:

> Pierre Smits <[hidden email]> writes:
>
> > I believe there are a few more where testing individual test-suites
> and/or
> > test-cases are dependent on data loaded in other test-suites and/or other
> > test-cases.
>
> I have the same experience.  Moreover another source of fragility is
> that tests depend on other tests within a single OFBiz “test-case”,
> meaning one test can depend on the data produced by another test.  This
> is acceptable for a “simple-method-test” because the order of execution
> is sequential and managed by OFBiz, but this is problematic for JUnit
> tests (Groovy, Java) because the order while being deterministic depends
> on the arbitrary order imposed by the JVM.
>
> For example I know for a fact that “QuoteTests.groovy” is suffering from
> that issue.
>
> > While I don't hear/read about failing testIntegration (except where code
> in
> > the base is faulty, not when test-suites/cases are faulty), I see
> following
> > failures in test executions in OFBiz against jdk11:
> >
> >
> >    1. Execution failed for task ':ofbiz --test component=webapp --test
> >    suitename=webapptests'.
> >    2. Execution failed for task ':ofbiz --test component=accounting
> --test
> >    suitename=invoicetest'.
> >    3. Execution failed for task ':ofbiz --test component=order --test
> >    suitename=ordertests'.
> >    4. Execution failed for task ':ofbiz --test component=product --test
> >    suitename=producttests'.
> >
> > Do we have these test failing also when doing the test execution against
> > jdk8?
> > *Caveat: I recently set this up, so there may still be some configuration
> > issues in the jdk11-test setup.. *
>
> I have just tested the “ordertests” test-suite with Icedtea 3.7 (jdk-8)
> and it is still failing, so it seems unrelated in that case.
>
> Thanks.
>
> --
> Mathieu Lirzin
> GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37
>

[Pierre Smits-3]

Hi Mathieu,

Is there a way to move this forward?

Best regards,

Pierre Smits

*Apache Trafodion <https://trafodion.apache.org>, Vice President*
*Apache Directory <https://directory.apache.org>, PMC Member*
Apache Incubator <https://incubator.apache.org>, committer
*Apache OFBiz <https://ofbiz.apache.org>, contributor (without privileges)
since 2008*
Apache Steve <https://steve.apache.org>, committer


On Sat, Apr 20, 2019 at 2:25 PM Pierre Smits <[hidden email]> wrote:

> Maybe we should move the load aspects regarding tests out of the test
> suite invocations altogether.
> The gradlew tasks states:
>
> task testIntegration(group: ofbizServer) {
>
> dependsOn 'ofbiz --test'
>
> description 'Run OFBiz integration tests; You must run loadAll before
> running this task'
>
> }
>
>
> IMO, loading test data could be part of the loadAll task.
>
>
> Best regards,
>
> Pierre Smits
>
> *Apache Trafodion <https://trafodion.apache.org>, Vice President*
> *Apache Directory <https://directory.apache.org>, PMC Member*
> Apache Incubator <https://incubator.apache.org>, committer
> *Apache OFBiz <https://ofbiz.apache.org>, contributor (without privileges)
> since 2008*
> Apache Steve <https://steve.apache.org>, committer
>
>
> On Sat, Apr 20, 2019 at 1:56 PM Mathieu Lirzin <[hidden email]>
> wrote:
>
>> Pierre Smits <[hidden email]> writes:
>>
>> > I believe there are a few more where testing individual test-suites
>> and/or
>> > test-cases are dependent on data loaded in other test-suites and/or
>> other
>> > test-cases.
>>
>> I have the same experience.  Moreover another source of fragility is
>> that tests depend on other tests within a single OFBiz “test-case”,
>> meaning one test can depend on the data produced by another test.  This
>> is acceptable for a “simple-method-test” because the order of execution
>> is sequential and managed by OFBiz, but this is problematic for JUnit
>> tests (Groovy, Java) because the order while being deterministic depends
>> on the arbitrary order imposed by the JVM.
>>
>> For example I know for a fact that “QuoteTests.groovy” is suffering from
>> that issue.
>>
>> > While I don't hear/read about failing testIntegration (except where
>> code in
>> > the base is faulty, not when test-suites/cases are faulty), I see
>> following
>> > failures in test executions in OFBiz against jdk11:
>> >
>> >
>> >    1. Execution failed for task ':ofbiz --test component=webapp --test
>> >    suitename=webapptests'.
>> >    2. Execution failed for task ':ofbiz --test component=accounting
>> --test
>> >    suitename=invoicetest'.
>> >    3. Execution failed for task ':ofbiz --test component=order --test
>> >    suitename=ordertests'.
>> >    4. Execution failed for task ':ofbiz --test component=product --test
>> >    suitename=producttests'.
>> >
>> > Do we have these test failing also when doing the test execution against
>> > jdk8?
>> > *Caveat: I recently set this up, so there may still be some
>> configuration
>> > issues in the jdk11-test setup.. *
>>
>> I have just tested the “ordertests” test-suite with Icedtea 3.7 (jdk-8)
>> and it is still failing, so it seems unrelated in that case.
>>
>> Thanks.
>>
>> --
>> Mathieu Lirzin
>> GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37
>>
>

[Jacques Le Roux]

Hi Suraj,

I think that, as suggested by Mathieu, in the meantime it's better to remove “OrderTests.groovy”

Because it could hide other issues else reported by Buildbot which is our last safeguard

Thanks

Jacques

Le 25/04/2019 à 10:52, Pierre Smits a écrit :

> Hi Mathieu,
>
> Is there a way to move this forward?
>
> Best regards,
>
> Pierre Smits
>
> *Apache Trafodion <https://trafodion.apache.org>, Vice President*
> *Apache Directory <https://directory.apache.org>, PMC Member*
> Apache Incubator <https://incubator.apache.org>, committer
> *Apache OFBiz <https://ofbiz.apache.org>, contributor (without privileges)
> since 2008*
> Apache Steve <https://steve.apache.org>, committer
>
>
> On Sat, Apr 20, 2019 at 2:25 PM Pierre Smits <[hidden email]> wrote:
>
>> Maybe we should move the load aspects regarding tests out of the test
>> suite invocations altogether.
>> The gradlew tasks states:
>>
>> task testIntegration(group: ofbizServer) {
>>
>> dependsOn 'ofbiz --test'
>>
>> description 'Run OFBiz integration tests; You must run loadAll before
>> running this task'
>>
>> }
>>
>>
>> IMO, loading test data could be part of the loadAll task.
>>
>>
>> Best regards,
>>
>> Pierre Smits
>>
>> *Apache Trafodion <https://trafodion.apache.org>, Vice President*
>> *Apache Directory <https://directory.apache.org>, PMC Member*
>> Apache Incubator <https://incubator.apache.org>, committer
>> *Apache OFBiz <https://ofbiz.apache.org>, contributor (without privileges)
>> since 2008*
>> Apache Steve <https://steve.apache.org>, committer
>>
>>
>> On Sat, Apr 20, 2019 at 1:56 PM Mathieu Lirzin <[hidden email]>
>> wrote:
>>
>>> Pierre Smits <[hidden email]> writes:
>>>
>>>> I believe there are a few more where testing individual test-suites
>>> and/or
>>>> test-cases are dependent on data loaded in other test-suites and/or
>>> other
>>>> test-cases.
>>> I have the same experience.  Moreover another source of fragility is
>>> that tests depend on other tests within a single OFBiz “test-case”,
>>> meaning one test can depend on the data produced by another test.  This
>>> is acceptable for a “simple-method-test” because the order of execution
>>> is sequential and managed by OFBiz, but this is problematic for JUnit
>>> tests (Groovy, Java) because the order while being deterministic depends
>>> on the arbitrary order imposed by the JVM.
>>>
>>> For example I know for a fact that “QuoteTests.groovy” is suffering from
>>> that issue.
>>>
>>>> While I don't hear/read about failing testIntegration (except where
>>> code in
>>>> the base is faulty, not when test-suites/cases are faulty), I see
>>> following
>>>> failures in test executions in OFBiz against jdk11:
>>>>
>>>>
>>>>     1. Execution failed for task ':ofbiz --test component=webapp --test
>>>>     suitename=webapptests'.
>>>>     2. Execution failed for task ':ofbiz --test component=accounting
>>> --test
>>>>     suitename=invoicetest'.
>>>>     3. Execution failed for task ':ofbiz --test component=order --test
>>>>     suitename=ordertests'.
>>>>     4. Execution failed for task ':ofbiz --test component=product --test
>>>>     suitename=producttests'.
>>>>
>>>> Do we have these test failing also when doing the test execution against
>>>> jdk8?
>>>> *Caveat: I recently set this up, so there may still be some
>>> configuration
>>>> issues in the jdk11-test setup.. *
>>> I have just tested the “ordertests” test-suite with Icedtea 3.7 (jdk-8)
>>> and it is still failing, so it seems unrelated in that case.
>>>
>>> Thanks.
>>>
>>> --
>>> Mathieu Lirzin
>>> GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37
>>>

[Suraj Khurana-2]

Sure Jacques,

I am into it today and if got nothing I will remove OrderTests.groovy

--
Best Regards,
Suraj Khurana







On Fri, Apr 26, 2019 at 7:27 PM Jacques Le Roux <
[hidden email]> wrote:

> Hi Suraj,
>
> I think that, as suggested by Mathieu, in the meantime it's better to
> remove “OrderTests.groovy”
>
> Because it could hide other issues else reported by Buildbot which is our
> last safeguard
>
> Thanks
>
> Jacques
>
> Le 25/04/2019 à 10:52, Pierre Smits a écrit :
> > Hi Mathieu,
> >
> > Is there a way to move this forward?
> >
> > Best regards,
> >
> > Pierre Smits
> >
> > *Apache Trafodion <https://trafodion.apache.org>, Vice President*
> > *Apache Directory <https://directory.apache.org>, PMC Member*
> > Apache Incubator <https://incubator.apache.org>, committer
> > *Apache OFBiz <https://ofbiz.apache.org>, contributor (without
> privileges)
> > since 2008*
> > Apache Steve <https://steve.apache.org>, committer
> >
> >
> > On Sat, Apr 20, 2019 at 2:25 PM Pierre Smits <[hidden email]>
> wrote:
> >
> >> Maybe we should move the load aspects regarding tests out of the test
> >> suite invocations altogether.
> >> The gradlew tasks states:
> >>
> >> task testIntegration(group: ofbizServer) {
> >>
> >> dependsOn 'ofbiz --test'
> >>
> >> description 'Run OFBiz integration tests; You must run loadAll before
> >> running this task'
> >>
> >> }
> >>
> >>
> >> IMO, loading test data could be part of the loadAll task.
> >>
> >>
> >> Best regards,
> >>
> >> Pierre Smits
> >>
> >> *Apache Trafodion <https://trafodion.apache.org>, Vice President*
> >> *Apache Directory <https://directory.apache.org>, PMC Member*
> >> Apache Incubator <https://incubator.apache.org>, committer
> >> *Apache OFBiz <https://ofbiz.apache.org>, contributor (without
> privileges)
> >> since 2008*
> >> Apache Steve <https://steve.apache.org>, committer
> >>
> >>
> >> On Sat, Apr 20, 2019 at 1:56 PM Mathieu Lirzin <
> [hidden email]>
> >> wrote:
> >>
> >>> Pierre Smits <[hidden email]> writes:
> >>>
> >>>> I believe there are a few more where testing individual test-suites
> >>> and/or
> >>>> test-cases are dependent on data loaded in other test-suites and/or
> >>> other
> >>>> test-cases.
> >>> I have the same experience.  Moreover another source of fragility is
> >>> that tests depend on other tests within a single OFBiz “test-case”,
> >>> meaning one test can depend on the data produced by another test.  This
> >>> is acceptable for a “simple-method-test” because the order of execution
> >>> is sequential and managed by OFBiz, but this is problematic for JUnit
> >>> tests (Groovy, Java) because the order while being deterministic
> depends
> >>> on the arbitrary order imposed by the JVM.
> >>>
> >>> For example I know for a fact that “QuoteTests.groovy” is suffering
> from
> >>> that issue.
> >>>
> >>>> While I don't hear/read about failing testIntegration (except where
> >>> code in
> >>>> the base is faulty, not when test-suites/cases are faulty), I see
> >>> following
> >>>> failures in test executions in OFBiz against jdk11:
> >>>>
> >>>>
> >>>>     1. Execution failed for task ':ofbiz --test component=webapp
> --test
> >>>>     suitename=webapptests'.
> >>>>     2. Execution failed for task ':ofbiz --test component=accounting
> >>> --test
> >>>>     suitename=invoicetest'.
> >>>>     3. Execution failed for task ':ofbiz --test component=order --test
> >>>>     suitename=ordertests'.
> >>>>     4. Execution failed for task ':ofbiz --test component=product
> --test
> >>>>     suitename=producttests'.
> >>>>
> >>>> Do we have these test failing also when doing the test execution
> against
> >>>> jdk8?
> >>>> *Caveat: I recently set this up, so there may still be some
> >>> configuration
> >>>> issues in the jdk11-test setup.. *
> >>> I have just tested the “ordertests” test-suite with Icedtea 3.7 (jdk-8)
> >>> and it is still failing, so it seems unrelated in that case.
> >>>
> >>> Thanks.
> >>>
> >>> --
> >>> Mathieu Lirzin
> >>> GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37
> >>>
>

[Suraj Khurana-2]

Hello team,

I have checked and found that there is a data dependency of
workEffortId=9000 in the test case which is available in plugins/projectmgr
component.

This was the main reason testIntegration was failing without having plugins
component. I will take care of it and add respective dependent data on
order test data file.

I think its making sense now and we don't need to revert now.

--
Best Regards,
Suraj Khurana






On Sat, Apr 27, 2019 at 10:14 AM Suraj Khurana <[hidden email]>
wrote:

> Sure Jacques,
>
> I am into it today and if got nothing I will remove OrderTests.groovy
>
> --
> Best Regards,
> Suraj Khurana
>
>
>
>
>
>
>
> On Fri, Apr 26, 2019 at 7:27 PM Jacques Le Roux <
> [hidden email]> wrote:
>
>> Hi Suraj,
>>
>> I think that, as suggested by Mathieu, in the meantime it's better to
>> remove “OrderTests.groovy”
>>
>> Because it could hide other issues else reported by Buildbot which is our
>> last safeguard
>>
>> Thanks
>>
>> Jacques
>>
>> Le 25/04/2019 à 10:52, Pierre Smits a écrit :
>> > Hi Mathieu,
>> >
>> > Is there a way to move this forward?
>> >
>> > Best regards,
>> >
>> > Pierre Smits
>> >
>> > *Apache Trafodion <https://trafodion.apache.org>, Vice President*
>> > *Apache Directory <https://directory.apache.org>, PMC Member*
>> > Apache Incubator <https://incubator.apache.org>, committer
>> > *Apache OFBiz <https://ofbiz.apache.org>, contributor (without
>> privileges)
>> > since 2008*
>> > Apache Steve <https://steve.apache.org>, committer
>> >
>> >
>> > On Sat, Apr 20, 2019 at 2:25 PM Pierre Smits <[hidden email]>
>> wrote:
>> >
>> >> Maybe we should move the load aspects regarding tests out of the test
>> >> suite invocations altogether.
>> >> The gradlew tasks states:
>> >>
>> >> task testIntegration(group: ofbizServer) {
>> >>
>> >> dependsOn 'ofbiz --test'
>> >>
>> >> description 'Run OFBiz integration tests; You must run loadAll before
>> >> running this task'
>> >>
>> >> }
>> >>
>> >>
>> >> IMO, loading test data could be part of the loadAll task.
>> >>
>> >>
>> >> Best regards,
>> >>
>> >> Pierre Smits
>> >>
>> >> *Apache Trafodion <https://trafodion.apache.org>, Vice President*
>> >> *Apache Directory <https://directory.apache.org>, PMC Member*
>> >> Apache Incubator <https://incubator.apache.org>, committer
>> >> *Apache OFBiz <https://ofbiz.apache.org>, contributor (without
>> privileges)
>> >> since 2008*
>> >> Apache Steve <https://steve.apache.org>, committer
>> >>
>> >>
>> >> On Sat, Apr 20, 2019 at 1:56 PM Mathieu Lirzin <
>> [hidden email]>
>> >> wrote:
>> >>
>> >>> Pierre Smits <[hidden email]> writes:
>> >>>
>> >>>> I believe there are a few more where testing individual test-suites
>> >>> and/or
>> >>>> test-cases are dependent on data loaded in other test-suites and/or
>> >>> other
>> >>>> test-cases.
>> >>> I have the same experience.  Moreover another source of fragility is
>> >>> that tests depend on other tests within a single OFBiz “test-case”,
>> >>> meaning one test can depend on the data produced by another test.
>> This
>> >>> is acceptable for a “simple-method-test” because the order of
>> execution
>> >>> is sequential and managed by OFBiz, but this is problematic for JUnit
>> >>> tests (Groovy, Java) because the order while being deterministic
>> depends
>> >>> on the arbitrary order imposed by the JVM.
>> >>>
>> >>> For example I know for a fact that “QuoteTests.groovy” is suffering
>> from
>> >>> that issue.
>> >>>
>> >>>> While I don't hear/read about failing testIntegration (except where
>> >>> code in
>> >>>> the base is faulty, not when test-suites/cases are faulty), I see
>> >>> following
>> >>>> failures in test executions in OFBiz against jdk11:
>> >>>>
>> >>>>
>> >>>>     1. Execution failed for task ':ofbiz --test component=webapp
>> --test
>> >>>>     suitename=webapptests'.
>> >>>>     2. Execution failed for task ':ofbiz --test component=accounting
>> >>> --test
>> >>>>     suitename=invoicetest'.
>> >>>>     3. Execution failed for task ':ofbiz --test component=order
>> --test
>> >>>>     suitename=ordertests'.
>> >>>>     4. Execution failed for task ':ofbiz --test component=product
>> --test
>> >>>>     suitename=producttests'.
>> >>>>
>> >>>> Do we have these test failing also when doing the test execution
>> against
>> >>>> jdk8?
>> >>>> *Caveat: I recently set this up, so there may still be some
>> >>> configuration
>> >>>> issues in the jdk11-test setup.. *
>> >>> I have just tested the “ordertests” test-suite with Icedtea 3.7
>> (jdk-8)
>> >>> and it is still failing, so it seems unrelated in that case.
>> >>>
>> >>> Thanks.
>> >>>
>> >>> --
>> >>> Mathieu Lirzin
>> >>> GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37
>> >>>
>>
>

[Jacques Le Roux]

Thanks Suraj,

Can't we avoid the duplicated data?

Jacques

Le 27/04/2019 à 15:17, Suraj Khurana a écrit :

> Hello team,
>
> I have checked and found that there is a data dependency of
> workEffortId=9000 in the test case which is available in plugins/projectmgr
> component.
>
> This was the main reason testIntegration was failing without having plugins
> component. I will take care of it and add respective dependent data on
> order test data file.
>
> I think its making sense now and we don't need to revert now.
>
> --
> Best Regards,
> Suraj Khurana
>
>
>
>
>
>
> On Sat, Apr 27, 2019 at 10:14 AM Suraj Khurana <[hidden email]>
> wrote:
>
>> Sure Jacques,
>>
>> I am into it today and if got nothing I will remove OrderTests.groovy
>>
>> --
>> Best Regards,
>> Suraj Khurana
>>
>>
>>
>>
>>
>>
>>
>> On Fri, Apr 26, 2019 at 7:27 PM Jacques Le Roux <
>> [hidden email]> wrote:
>>
>>> Hi Suraj,
>>>
>>> I think that, as suggested by Mathieu, in the meantime it's better to
>>> remove “OrderTests.groovy”
>>>
>>> Because it could hide other issues else reported by Buildbot which is our
>>> last safeguard
>>>
>>> Thanks
>>>
>>> Jacques
>>>
>>> Le 25/04/2019 à 10:52, Pierre Smits a écrit :
>>>> Hi Mathieu,
>>>>
>>>> Is there a way to move this forward?
>>>>
>>>> Best regards,
>>>>
>>>> Pierre Smits
>>>>
>>>> *Apache Trafodion <https://trafodion.apache.org>, Vice President*
>>>> *Apache Directory <https://directory.apache.org>, PMC Member*
>>>> Apache Incubator <https://incubator.apache.org>, committer
>>>> *Apache OFBiz <https://ofbiz.apache.org>, contributor (without
>>> privileges)
>>>> since 2008*
>>>> Apache Steve <https://steve.apache.org>, committer
>>>>
>>>>
>>>> On Sat, Apr 20, 2019 at 2:25 PM Pierre Smits <[hidden email]>
>>> wrote:
>>>>> Maybe we should move the load aspects regarding tests out of the test
>>>>> suite invocations altogether.
>>>>> The gradlew tasks states:
>>>>>
>>>>> task testIntegration(group: ofbizServer) {
>>>>>
>>>>> dependsOn 'ofbiz --test'
>>>>>
>>>>> description 'Run OFBiz integration tests; You must run loadAll before
>>>>> running this task'
>>>>>
>>>>> }
>>>>>
>>>>>
>>>>> IMO, loading test data could be part of the loadAll task.
>>>>>
>>>>>
>>>>> Best regards,
>>>>>
>>>>> Pierre Smits
>>>>>
>>>>> *Apache Trafodion <https://trafodion.apache.org>, Vice President*
>>>>> *Apache Directory <https://directory.apache.org>, PMC Member*
>>>>> Apache Incubator <https://incubator.apache.org>, committer
>>>>> *Apache OFBiz <https://ofbiz.apache.org>, contributor (without
>>> privileges)
>>>>> since 2008*
>>>>> Apache Steve <https://steve.apache.org>, committer
>>>>>
>>>>>
>>>>> On Sat, Apr 20, 2019 at 1:56 PM Mathieu Lirzin <
>>> [hidden email]>
>>>>> wrote:
>>>>>
>>>>>> Pierre Smits <[hidden email]> writes:
>>>>>>
>>>>>>> I believe there are a few more where testing individual test-suites
>>>>>> and/or
>>>>>>> test-cases are dependent on data loaded in other test-suites and/or
>>>>>> other
>>>>>>> test-cases.
>>>>>> I have the same experience.  Moreover another source of fragility is
>>>>>> that tests depend on other tests within a single OFBiz “test-case”,
>>>>>> meaning one test can depend on the data produced by another test.
>>> This
>>>>>> is acceptable for a “simple-method-test” because the order of
>>> execution
>>>>>> is sequential and managed by OFBiz, but this is problematic for JUnit
>>>>>> tests (Groovy, Java) because the order while being deterministic
>>> depends
>>>>>> on the arbitrary order imposed by the JVM.
>>>>>>
>>>>>> For example I know for a fact that “QuoteTests.groovy” is suffering
>>> from
>>>>>> that issue.
>>>>>>
>>>>>>> While I don't hear/read about failing testIntegration (except where
>>>>>> code in
>>>>>>> the base is faulty, not when test-suites/cases are faulty), I see
>>>>>> following
>>>>>>> failures in test executions in OFBiz against jdk11:
>>>>>>>
>>>>>>>
>>>>>>>      1. Execution failed for task ':ofbiz --test component=webapp
>>> --test
>>>>>>>      suitename=webapptests'.
>>>>>>>      2. Execution failed for task ':ofbiz --test component=accounting
>>>>>> --test
>>>>>>>      suitename=invoicetest'.
>>>>>>>      3. Execution failed for task ':ofbiz --test component=order
>>> --test
>>>>>>>      suitename=ordertests'.
>>>>>>>      4. Execution failed for task ':ofbiz --test component=product
>>> --test
>>>>>>>      suitename=producttests'.
>>>>>>>
>>>>>>> Do we have these test failing also when doing the test execution
>>> against
>>>>>>> jdk8?
>>>>>>> *Caveat: I recently set this up, so there may still be some
>>>>>> configuration
>>>>>>> issues in the jdk11-test setup.. *
>>>>>> I have just tested the “ordertests” test-suite with Icedtea 3.7
>>> (jdk-8)
>>>>>> and it is still failing, so it seems unrelated in that case.
>>>>>>
>>>>>> Thanks.
>>>>>>
>>>>>> --
>>>>>> Mathieu Lirzin
>>>>>> GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37
>>>>>>

[Pierre Smits-3]

Hi Jacques, all,

Currently, we can't avoid having duplicates in test data when we load such
data just before the execution a test-suite/test-case. We should not
concern ourselves to much with this. After all it is just data for test,
and reloading a few duplicates should not be regarded as a major issue.

However, if the community is adamantly set on removing such duplicates,
then it should work on having test-data being loaded before any and all
test-suites/test-cases gets executed. IMO this involves moving test-data
from within the testdef folder (like in the order component) to the data
folder of the component and having a separate loadTestData task.

Best regards,

Pierre Smits

*Apache Trafodion <https://trafodion.apache.org>, Vice President*
*Apache Directory <https://directory.apache.org>, PMC Member*
Apache Incubator <https://incubator.apache.org>, committer
*Apache OFBiz <https://ofbiz.apache.org>, contributor (without privileges)
since 2008*
Apache Steve <https://steve.apache.org>, committer


On Sat, Apr 27, 2019 at 3:38 PM Jacques Le Roux <
[hidden email]> wrote:

> Thanks Suraj,
>
> Can't we avoid the duplicated data?
>
> Jacques
>
> Le 27/04/2019 à 15:17, Suraj Khurana a écrit :
> > Hello team,
> >
> > I have checked and found that there is a data dependency of
> > workEffortId=9000 in the test case which is available in
> plugins/projectmgr
> > component.
> >
> > This was the main reason testIntegration was failing without having
> plugins
> > component. I will take care of it and add respective dependent data on
> > order test data file.
> >
> > I think its making sense now and we don't need to revert now.
> >
> > --
> > Best Regards,
> > Suraj Khurana
> >
> >
> >
> >
> >
> >
> > On Sat, Apr 27, 2019 at 10:14 AM Suraj Khurana <[hidden email]>
> > wrote:
> >
> >> Sure Jacques,
> >>
> >> I am into it today and if got nothing I will remove OrderTests.groovy
> >>
> >> --
> >> Best Regards,
> >> Suraj Khurana
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> On Fri, Apr 26, 2019 at 7:27 PM Jacques Le Roux <
> >> [hidden email]> wrote:
> >>
> >>> Hi Suraj,
> >>>
> >>> I think that, as suggested by Mathieu, in the meantime it's better to
> >>> remove “OrderTests.groovy”
> >>>
> >>> Because it could hide other issues else reported by Buildbot which is
> our
> >>> last safeguard
> >>>
> >>> Thanks
> >>>
> >>> Jacques
> >>>
> >>> Le 25/04/2019 à 10:52, Pierre Smits a écrit :
> >>>> Hi Mathieu,
> >>>>
> >>>> Is there a way to move this forward?
> >>>>
> >>>> Best regards,
> >>>>
> >>>> Pierre Smits
> >>>>
> >>>> *Apache Trafodion <https://trafodion.apache.org>, Vice President*
> >>>> *Apache Directory <https://directory.apache.org>, PMC Member*
> >>>> Apache Incubator <https://incubator.apache.org>, committer
> >>>> *Apache OFBiz <https://ofbiz.apache.org>, contributor (without
> >>> privileges)
> >>>> since 2008*
> >>>> Apache Steve <https://steve.apache.org>, committer
> >>>>
> >>>>
> >>>> On Sat, Apr 20, 2019 at 2:25 PM Pierre Smits <[hidden email]>
> >>> wrote:
> >>>>> Maybe we should move the load aspects regarding tests out of the test
> >>>>> suite invocations altogether.
> >>>>> The gradlew tasks states:
> >>>>>
> >>>>> task testIntegration(group: ofbizServer) {
> >>>>>
> >>>>> dependsOn 'ofbiz --test'
> >>>>>
> >>>>> description 'Run OFBiz integration tests; You must run loadAll before
> >>>>> running this task'
> >>>>>
> >>>>> }
> >>>>>
> >>>>>
> >>>>> IMO, loading test data could be part of the loadAll task.
> >>>>>
> >>>>>
> >>>>> Best regards,
> >>>>>
> >>>>> Pierre Smits
> >>>>>
> >>>>> *Apache Trafodion <https://trafodion.apache.org>, Vice President*
> >>>>> *Apache Directory <https://directory.apache.org>, PMC Member*
> >>>>> Apache Incubator <https://incubator.apache.org>, committer
> >>>>> *Apache OFBiz <https://ofbiz.apache.org>, contributor (without
> >>> privileges)
> >>>>> since 2008*
> >>>>> Apache Steve <https://steve.apache.org>, committer
> >>>>>
> >>>>>
> >>>>> On Sat, Apr 20, 2019 at 1:56 PM Mathieu Lirzin <
> >>> [hidden email]>
> >>>>> wrote:
> >>>>>
> >>>>>> Pierre Smits <[hidden email]> writes:
> >>>>>>
> >>>>>>> I believe there are a few more where testing individual test-suites
> >>>>>> and/or
> >>>>>>> test-cases are dependent on data loaded in other test-suites and/or
> >>>>>> other
> >>>>>>> test-cases.
> >>>>>> I have the same experience.  Moreover another source of fragility is
> >>>>>> that tests depend on other tests within a single OFBiz “test-case”,
> >>>>>> meaning one test can depend on the data produced by another test.
> >>> This
> >>>>>> is acceptable for a “simple-method-test” because the order of
> >>> execution
> >>>>>> is sequential and managed by OFBiz, but this is problematic for
> JUnit
> >>>>>> tests (Groovy, Java) because the order while being deterministic
> >>> depends
> >>>>>> on the arbitrary order imposed by the JVM.
> >>>>>>
> >>>>>> For example I know for a fact that “QuoteTests.groovy” is suffering
> >>> from
> >>>>>> that issue.
> >>>>>>
> >>>>>>> While I don't hear/read about failing testIntegration (except where
> >>>>>> code in
> >>>>>>> the base is faulty, not when test-suites/cases are faulty), I see
> >>>>>> following
> >>>>>>> failures in test executions in OFBiz against jdk11:
> >>>>>>>
> >>>>>>>
> >>>>>>>      1. Execution failed for task ':ofbiz --test component=webapp
> >>> --test
> >>>>>>>      suitename=webapptests'.
> >>>>>>>      2. Execution failed for task ':ofbiz --test
> component=accounting
> >>>>>> --test
> >>>>>>>      suitename=invoicetest'.
> >>>>>>>      3. Execution failed for task ':ofbiz --test component=order
> >>> --test
> >>>>>>>      suitename=ordertests'.
> >>>>>>>      4. Execution failed for task ':ofbiz --test component=product
> >>> --test
> >>>>>>>      suitename=producttests'.
> >>>>>>>
> >>>>>>> Do we have these test failing also when doing the test execution
> >>> against
> >>>>>>> jdk8?
> >>>>>>> *Caveat: I recently set this up, so there may still be some
> >>>>>> configuration
> >>>>>>> issues in the jdk11-test setup.. *
> >>>>>> I have just tested the “ordertests” test-suite with Icedtea 3.7
> >>> (jdk-8)
> >>>>>> and it is still failing, so it seems unrelated in that case.
> >>>>>>
> >>>>>> Thanks.
> >>>>>>
> >>>>>> --
> >>>>>> Mathieu Lirzin
> >>>>>> GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37
> >>>>>>
>

[Jacques Le Roux]

Yes you are right Pierre, it's not a worry when it's only for test, missed that

Jacques

Le 28/04/2019 à 09:52, Pierre Smits a écrit :

> Hi Jacques, all,
>
> Currently, we can't avoid having duplicates in test data when we load such
> data just before the execution a test-suite/test-case. We should not
> concern ourselves to much with this. After all it is just data for test,
> and reloading a few duplicates should not be regarded as a major issue.
>
> However, if the community is adamantly set on removing such duplicates,
> then it should work on having test-data being loaded before any and all
> test-suites/test-cases gets executed. IMO this involves moving test-data
> from within the testdef folder (like in the order component) to the data
> folder of the component and having a separate loadTestData task.
>
> Best regards,
>
> Pierre Smits
>
> *Apache Trafodion <https://trafodion.apache.org>, Vice President*
> *Apache Directory <https://directory.apache.org>, PMC Member*
> Apache Incubator <https://incubator.apache.org>, committer
> *Apache OFBiz <https://ofbiz.apache.org>, contributor (without privileges)
> since 2008*
> Apache Steve <https://steve.apache.org>, committer
>
>
> On Sat, Apr 27, 2019 at 3:38 PM Jacques Le Roux <
> [hidden email]> wrote:
>
>> Thanks Suraj,
>>
>> Can't we avoid the duplicated data?
>>
>> Jacques
>>
>> Le 27/04/2019 à 15:17, Suraj Khurana a écrit :
>>> Hello team,
>>>
>>> I have checked and found that there is a data dependency of
>>> workEffortId=9000 in the test case which is available in
>> plugins/projectmgr
>>> component.
>>>
>>> This was the main reason testIntegration was failing without having
>> plugins
>>> component. I will take care of it and add respective dependent data on
>>> order test data file.
>>>
>>> I think its making sense now and we don't need to revert now.
>>>
>>> --
>>> Best Regards,
>>> Suraj Khurana
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Sat, Apr 27, 2019 at 10:14 AM Suraj Khurana <[hidden email]>
>>> wrote:
>>>
>>>> Sure Jacques,
>>>>
>>>> I am into it today and if got nothing I will remove OrderTests.groovy
>>>>
>>>> --
>>>> Best Regards,
>>>> Suraj Khurana
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Fri, Apr 26, 2019 at 7:27 PM Jacques Le Roux <
>>>> [hidden email]> wrote:
>>>>
>>>>> Hi Suraj,
>>>>>
>>>>> I think that, as suggested by Mathieu, in the meantime it's better to
>>>>> remove “OrderTests.groovy”
>>>>>
>>>>> Because it could hide other issues else reported by Buildbot which is
>> our
>>>>> last safeguard
>>>>>
>>>>> Thanks
>>>>>
>>>>> Jacques
>>>>>
>>>>> Le 25/04/2019 à 10:52, Pierre Smits a écrit :
>>>>>> Hi Mathieu,
>>>>>>
>>>>>> Is there a way to move this forward?
>>>>>>
>>>>>> Best regards,
>>>>>>
>>>>>> Pierre Smits
>>>>>>
>>>>>> *Apache Trafodion <https://trafodion.apache.org>, Vice President*
>>>>>> *Apache Directory <https://directory.apache.org>, PMC Member*
>>>>>> Apache Incubator <https://incubator.apache.org>, committer
>>>>>> *Apache OFBiz <https://ofbiz.apache.org>, contributor (without
>>>>> privileges)
>>>>>> since 2008*
>>>>>> Apache Steve <https://steve.apache.org>, committer
>>>>>>
>>>>>>
>>>>>> On Sat, Apr 20, 2019 at 2:25 PM Pierre Smits <[hidden email]>
>>>>> wrote:
>>>>>>> Maybe we should move the load aspects regarding tests out of the test
>>>>>>> suite invocations altogether.
>>>>>>> The gradlew tasks states:
>>>>>>>
>>>>>>> task testIntegration(group: ofbizServer) {
>>>>>>>
>>>>>>> dependsOn 'ofbiz --test'
>>>>>>>
>>>>>>> description 'Run OFBiz integration tests; You must run loadAll before
>>>>>>> running this task'
>>>>>>>
>>>>>>> }
>>>>>>>
>>>>>>>
>>>>>>> IMO, loading test data could be part of the loadAll task.
>>>>>>>
>>>>>>>
>>>>>>> Best regards,
>>>>>>>
>>>>>>> Pierre Smits
>>>>>>>
>>>>>>> *Apache Trafodion <https://trafodion.apache.org>, Vice President*
>>>>>>> *Apache Directory <https://directory.apache.org>, PMC Member*
>>>>>>> Apache Incubator <https://incubator.apache.org>, committer
>>>>>>> *Apache OFBiz <https://ofbiz.apache.org>, contributor (without
>>>>> privileges)
>>>>>>> since 2008*
>>>>>>> Apache Steve <https://steve.apache.org>, committer
>>>>>>>
>>>>>>>
>>>>>>> On Sat, Apr 20, 2019 at 1:56 PM Mathieu Lirzin <
>>>>> [hidden email]>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Pierre Smits <[hidden email]> writes:
>>>>>>>>
>>>>>>>>> I believe there are a few more where testing individual test-suites
>>>>>>>> and/or
>>>>>>>>> test-cases are dependent on data loaded in other test-suites and/or
>>>>>>>> other
>>>>>>>>> test-cases.
>>>>>>>> I have the same experience.  Moreover another source of fragility is
>>>>>>>> that tests depend on other tests within a single OFBiz “test-case”,
>>>>>>>> meaning one test can depend on the data produced by another test.
>>>>> This
>>>>>>>> is acceptable for a “simple-method-test” because the order of
>>>>> execution
>>>>>>>> is sequential and managed by OFBiz, but this is problematic for
>> JUnit
>>>>>>>> tests (Groovy, Java) because the order while being deterministic
>>>>> depends
>>>>>>>> on the arbitrary order imposed by the JVM.
>>>>>>>>
>>>>>>>> For example I know for a fact that “QuoteTests.groovy” is suffering
>>>>> from
>>>>>>>> that issue.
>>>>>>>>
>>>>>>>>> While I don't hear/read about failing testIntegration (except where
>>>>>>>> code in
>>>>>>>>> the base is faulty, not when test-suites/cases are faulty), I see
>>>>>>>> following
>>>>>>>>> failures in test executions in OFBiz against jdk11:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>       1. Execution failed for task ':ofbiz --test component=webapp
>>>>> --test
>>>>>>>>>       suitename=webapptests'.
>>>>>>>>>       2. Execution failed for task ':ofbiz --test
>> component=accounting
>>>>>>>> --test
>>>>>>>>>       suitename=invoicetest'.
>>>>>>>>>       3. Execution failed for task ':ofbiz --test component=order
>>>>> --test
>>>>>>>>>       suitename=ordertests'.
>>>>>>>>>       4. Execution failed for task ':ofbiz --test component=product
>>>>> --test
>>>>>>>>>       suitename=producttests'.
>>>>>>>>>
>>>>>>>>> Do we have these test failing also when doing the test execution
>>>>> against
>>>>>>>>> jdk8?
>>>>>>>>> *Caveat: I recently set this up, so there may still be some
>>>>>>>> configuration
>>>>>>>>> issues in the jdk11-test setup.. *
>>>>>>>> I have just tested the “ordertests” test-suite with Icedtea 3.7
>>>>> (jdk-8)
>>>>>>>> and it is still failing, so it seems unrelated in that case.
>>>>>>>>
>>>>>>>> Thanks.
>>>>>>>>
>>>>>>>> --
>>>>>>>> Mathieu Lirzin
>>>>>>>> GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37
>>>>>>>>

[Pierre Smits-3]

Hi Jacques,

What are you suggesting? More duplicates? Or having all test data loaded
before any test starts?

Best regards,

Pierre Smits

*Apache Trafodion <https://trafodion.apache.org>, Vice President*
*Apache Directory <https://directory.apache.org>, PMC Member*
Apache Incubator <https://incubator.apache.org>, committer
*Apache OFBiz <https://ofbiz.apache.org>, contributor (without privileges)
since 2008*
Apache Steve <https://steve.apache.org>, committer


On Sun, Apr 28, 2019 at 2:29 PM Jacques Le Roux <
[hidden email]> wrote:

> Yes you are right Pierre, it's not a worry when it's only for test, missed
> that
>
> Jacques
>
> Le 28/04/2019 à 09:52, Pierre Smits a écrit :
> > Hi Jacques, all,
> >
> > Currently, we can't avoid having duplicates in test data when we load
> such
> > data just before the execution a test-suite/test-case. We should not
> > concern ourselves to much with this. After all it is just data for test,
> > and reloading a few duplicates should not be regarded as a major issue.
> >
> > However, if the community is adamantly set on removing such duplicates,
> > then it should work on having test-data being loaded before any and all
> > test-suites/test-cases gets executed. IMO this involves moving test-data
> > from within the testdef folder (like in the order component) to the data
> > folder of the component and having a separate loadTestData task.
> >
> > Best regards,
> >
> > Pierre Smits
> >
> > *Apache Trafodion <https://trafodion.apache.org>, Vice President*
> > *Apache Directory <https://directory.apache.org>, PMC Member*
> > Apache Incubator <https://incubator.apache.org>, committer
> > *Apache OFBiz <https://ofbiz.apache.org>, contributor (without
> privileges)
> > since 2008*
> > Apache Steve <https://steve.apache.org>, committer
> >
> >
> > On Sat, Apr 27, 2019 at 3:38 PM Jacques Le Roux <
> > [hidden email]> wrote:
> >
> >> Thanks Suraj,
> >>
> >> Can't we avoid the duplicated data?
> >>
> >> Jacques
> >>
> >> Le 27/04/2019 à 15:17, Suraj Khurana a écrit :
> >>> Hello team,
> >>>
> >>> I have checked and found that there is a data dependency of
> >>> workEffortId=9000 in the test case which is available in
> >> plugins/projectmgr
> >>> component.
> >>>
> >>> This was the main reason testIntegration was failing without having
> >> plugins
> >>> component. I will take care of it and add respective dependent data on
> >>> order test data file.
> >>>
> >>> I think its making sense now and we don't need to revert now.
> >>>
> >>> --
> >>> Best Regards,
> >>> Suraj Khurana
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> On Sat, Apr 27, 2019 at 10:14 AM Suraj Khurana <
> [hidden email]>
> >>> wrote:
> >>>
> >>>> Sure Jacques,
> >>>>
> >>>> I am into it today and if got nothing I will remove OrderTests.groovy
> >>>>
> >>>> --
> >>>> Best Regards,
> >>>> Suraj Khurana
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> On Fri, Apr 26, 2019 at 7:27 PM Jacques Le Roux <
> >>>> [hidden email]> wrote:
> >>>>
> >>>>> Hi Suraj,
> >>>>>
> >>>>> I think that, as suggested by Mathieu, in the meantime it's better to
> >>>>> remove “OrderTests.groovy”
> >>>>>
> >>>>> Because it could hide other issues else reported by Buildbot which is
> >> our
> >>>>> last safeguard
> >>>>>
> >>>>> Thanks
> >>>>>
> >>>>> Jacques
> >>>>>
> >>>>> Le 25/04/2019 à 10:52, Pierre Smits a écrit :
> >>>>>> Hi Mathieu,
> >>>>>>
> >>>>>> Is there a way to move this forward?
> >>>>>>
> >>>>>> Best regards,
> >>>>>>
> >>>>>> Pierre Smits
> >>>>>>
> >>>>>> *Apache Trafodion <https://trafodion.apache.org>, Vice President*
> >>>>>> *Apache Directory <https://directory.apache.org>, PMC Member*
> >>>>>> Apache Incubator <https://incubator.apache.org>, committer
> >>>>>> *Apache OFBiz <https://ofbiz.apache.org>, contributor (without
> >>>>> privileges)
> >>>>>> since 2008*
> >>>>>> Apache Steve <https://steve.apache.org>, committer
> >>>>>>
> >>>>>>
> >>>>>> On Sat, Apr 20, 2019 at 2:25 PM Pierre Smits <
> [hidden email]>
> >>>>> wrote:
> >>>>>>> Maybe we should move the load aspects regarding tests out of the
> test
> >>>>>>> suite invocations altogether.
> >>>>>>> The gradlew tasks states:
> >>>>>>>
> >>>>>>> task testIntegration(group: ofbizServer) {
> >>>>>>>
> >>>>>>> dependsOn 'ofbiz --test'
> >>>>>>>
> >>>>>>> description 'Run OFBiz integration tests; You must run loadAll
> before
> >>>>>>> running this task'
> >>>>>>>
> >>>>>>> }
> >>>>>>>
> >>>>>>>
> >>>>>>> IMO, loading test data could be part of the loadAll task.
> >>>>>>>
> >>>>>>>
> >>>>>>> Best regards,
> >>>>>>>
> >>>>>>> Pierre Smits
> >>>>>>>
> >>>>>>> *Apache Trafodion <https://trafodion.apache.org>, Vice President*
> >>>>>>> *Apache Directory <https://directory.apache.org>, PMC Member*
> >>>>>>> Apache Incubator <https://incubator.apache.org>, committer
> >>>>>>> *Apache OFBiz <https://ofbiz.apache.org>, contributor (without
> >>>>> privileges)
> >>>>>>> since 2008*
> >>>>>>> Apache Steve <https://steve.apache.org>, committer
> >>>>>>>
> >>>>>>>
> >>>>>>> On Sat, Apr 20, 2019 at 1:56 PM Mathieu Lirzin <
> >>>>> [hidden email]>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Pierre Smits <[hidden email]> writes:
> >>>>>>>>
> >>>>>>>>> I believe there are a few more where testing individual
> test-suites
> >>>>>>>> and/or
> >>>>>>>>> test-cases are dependent on data loaded in other test-suites
> and/or
> >>>>>>>> other
> >>>>>>>>> test-cases.
> >>>>>>>> I have the same experience.  Moreover another source of fragility
> is
> >>>>>>>> that tests depend on other tests within a single OFBiz
> “test-case”,
> >>>>>>>> meaning one test can depend on the data produced by another test.
> >>>>> This
> >>>>>>>> is acceptable for a “simple-method-test” because the order of
> >>>>> execution
> >>>>>>>> is sequential and managed by OFBiz, but this is problematic for
> >> JUnit
> >>>>>>>> tests (Groovy, Java) because the order while being deterministic
> >>>>> depends
> >>>>>>>> on the arbitrary order imposed by the JVM.
> >>>>>>>>
> >>>>>>>> For example I know for a fact that “QuoteTests.groovy” is
> suffering
> >>>>> from
> >>>>>>>> that issue.
> >>>>>>>>
> >>>>>>>>> While I don't hear/read about failing testIntegration (except
> where
> >>>>>>>> code in
> >>>>>>>>> the base is faulty, not when test-suites/cases are faulty), I see
> >>>>>>>> following
> >>>>>>>>> failures in test executions in OFBiz against jdk11:
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>       1. Execution failed for task ':ofbiz --test
> component=webapp
> >>>>> --test
> >>>>>>>>>       suitename=webapptests'.
> >>>>>>>>>       2. Execution failed for task ':ofbiz --test
> >> component=accounting
> >>>>>>>> --test
> >>>>>>>>>       suitename=invoicetest'.
> >>>>>>>>>       3. Execution failed for task ':ofbiz --test component=order
> >>>>> --test
> >>>>>>>>>       suitename=ordertests'.
> >>>>>>>>>       4. Execution failed for task ':ofbiz --test
> component=product
> >>>>> --test
> >>>>>>>>>       suitename=producttests'.
> >>>>>>>>>
> >>>>>>>>> Do we have these test failing also when doing the test execution
> >>>>> against
> >>>>>>>>> jdk8?
> >>>>>>>>> *Caveat: I recently set this up, so there may still be some
> >>>>>>>> configuration
> >>>>>>>>> issues in the jdk11-test setup.. *
> >>>>>>>> I have just tested the “ordertests” test-suite with Icedtea 3.7
> >>>>> (jdk-8)
> >>>>>>>> and it is still failing, so it seems unrelated in that case.
> >>>>>>>>
> >>>>>>>> Thanks.
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> Mathieu Lirzin
> >>>>>>>> GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37
> >>>>>>>>
>

[Jacques Le Roux]

Actually I rarely use integration tests independently. Most of the time I simply use a batch file which does:

C:\projectsASF\ofbiz>type test.bat
gup cleanAll eclipse loadAll testIntegration
C:\projectsASF\ofbiz>type gup.bat
svn up
cd plugins
svn up
cd ..
gradlew  %*
C:\projectsASF\ofbiz>

In other words to test I barely do the same than the Buildbot script does. It's the only way to be sure of what your are doing. That's pragmatic and
it works, though it's slow.

So in theory I prefer "having all test data loaded before any test starts"

But that's not contradictory to have data duplicates when you want to run a test or a suite alone.

Obviously if you want to be pragmatic (ie often meaning fast) you maybe need duplicates.

Not sure what we achieve with this discussion :D

Jacques

Le 28/04/2019 à 15:14, Pierre Smits a écrit :

> Hi Jacques,
>
> What are you suggesting? More duplicates? Or having all test data loaded
> before any test starts?
>
> Best regards,
>
> Pierre Smits
>
> *Apache Trafodion <https://trafodion.apache.org>, Vice President*
> *Apache Directory <https://directory.apache.org>, PMC Member*
> Apache Incubator <https://incubator.apache.org>, committer
> *Apache OFBiz <https://ofbiz.apache.org>, contributor (without privileges)
> since 2008*
> Apache Steve <https://steve.apache.org>, committer
>
>
> On Sun, Apr 28, 2019 at 2:29 PM Jacques Le Roux <
> [hidden email]> wrote:
>
>> Yes you are right Pierre, it's not a worry when it's only for test, missed
>> that
>>
>> Jacques
>>
>> Le 28/04/2019 à 09:52, Pierre Smits a écrit :
>>> Hi Jacques, all,
>>>
>>> Currently, we can't avoid having duplicates in test data when we load
>> such
>>> data just before the execution a test-suite/test-case. We should not
>>> concern ourselves to much with this. After all it is just data for test,
>>> and reloading a few duplicates should not be regarded as a major issue.
>>>
>>> However, if the community is adamantly set on removing such duplicates,
>>> then it should work on having test-data being loaded before any and all
>>> test-suites/test-cases gets executed. IMO this involves moving test-data
>>> from within the testdef folder (like in the order component) to the data
>>> folder of the component and having a separate loadTestData task.
>>>
>>> Best regards,
>>>
>>> Pierre Smits
>>>
>>> *Apache Trafodion <https://trafodion.apache.org>, Vice President*
>>> *Apache Directory <https://directory.apache.org>, PMC Member*
>>> Apache Incubator <https://incubator.apache.org>, committer
>>> *Apache OFBiz <https://ofbiz.apache.org>, contributor (without
>> privileges)
>>> since 2008*
>>> Apache Steve <https://steve.apache.org>, committer
>>>
>>>
>>> On Sat, Apr 27, 2019 at 3:38 PM Jacques Le Roux <
>>> [hidden email]> wrote:
>>>
>>>> Thanks Suraj,
>>>>
>>>> Can't we avoid the duplicated data?
>>>>
>>>> Jacques
>>>>
>>>> Le 27/04/2019 à 15:17, Suraj Khurana a écrit :
>>>>> Hello team,
>>>>>
>>>>> I have checked and found that there is a data dependency of
>>>>> workEffortId=9000 in the test case which is available in
>>>> plugins/projectmgr
>>>>> component.
>>>>>
>>>>> This was the main reason testIntegration was failing without having
>>>> plugins
>>>>> component. I will take care of it and add respective dependent data on
>>>>> order test data file.
>>>>>
>>>>> I think its making sense now and we don't need to revert now.
>>>>>
>>>>> --
>>>>> Best Regards,
>>>>> Suraj Khurana
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Sat, Apr 27, 2019 at 10:14 AM Suraj Khurana <
>> [hidden email]>
>>>>> wrote:
>>>>>
>>>>>> Sure Jacques,
>>>>>>
>>>>>> I am into it today and if got nothing I will remove OrderTests.groovy
>>>>>>
>>>>>> --
>>>>>> Best Regards,
>>>>>> Suraj Khurana
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, Apr 26, 2019 at 7:27 PM Jacques Le Roux <
>>>>>> [hidden email]> wrote:
>>>>>>
>>>>>>> Hi Suraj,
>>>>>>>
>>>>>>> I think that, as suggested by Mathieu, in the meantime it's better to
>>>>>>> remove “OrderTests.groovy”
>>>>>>>
>>>>>>> Because it could hide other issues else reported by Buildbot which is
>>>> our
>>>>>>> last safeguard
>>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>> Jacques
>>>>>>>
>>>>>>> Le 25/04/2019 à 10:52, Pierre Smits a écrit :
>>>>>>>> Hi Mathieu,
>>>>>>>>
>>>>>>>> Is there a way to move this forward?
>>>>>>>>
>>>>>>>> Best regards,
>>>>>>>>
>>>>>>>> Pierre Smits
>>>>>>>>
>>>>>>>> *Apache Trafodion <https://trafodion.apache.org>, Vice President*
>>>>>>>> *Apache Directory <https://directory.apache.org>, PMC Member*
>>>>>>>> Apache Incubator <https://incubator.apache.org>, committer
>>>>>>>> *Apache OFBiz <https://ofbiz.apache.org>, contributor (without
>>>>>>> privileges)
>>>>>>>> since 2008*
>>>>>>>> Apache Steve <https://steve.apache.org>, committer
>>>>>>>>
>>>>>>>>
>>>>>>>> On Sat, Apr 20, 2019 at 2:25 PM Pierre Smits <
>> [hidden email]>
>>>>>>> wrote:
>>>>>>>>> Maybe we should move the load aspects regarding tests out of the
>> test
>>>>>>>>> suite invocations altogether.
>>>>>>>>> The gradlew tasks states:
>>>>>>>>>
>>>>>>>>> task testIntegration(group: ofbizServer) {
>>>>>>>>>
>>>>>>>>> dependsOn 'ofbiz --test'
>>>>>>>>>
>>>>>>>>> description 'Run OFBiz integration tests; You must run loadAll
>> before
>>>>>>>>> running this task'
>>>>>>>>>
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> IMO, loading test data could be part of the loadAll task.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Best regards,
>>>>>>>>>
>>>>>>>>> Pierre Smits
>>>>>>>>>
>>>>>>>>> *Apache Trafodion <https://trafodion.apache.org>, Vice President*
>>>>>>>>> *Apache Directory <https://directory.apache.org>, PMC Member*
>>>>>>>>> Apache Incubator <https://incubator.apache.org>, committer
>>>>>>>>> *Apache OFBiz <https://ofbiz.apache.org>, contributor (without
>>>>>>> privileges)
>>>>>>>>> since 2008*
>>>>>>>>> Apache Steve <https://steve.apache.org>, committer
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Sat, Apr 20, 2019 at 1:56 PM Mathieu Lirzin <
>>>>>>> [hidden email]>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Pierre Smits <[hidden email]> writes:
>>>>>>>>>>
>>>>>>>>>>> I believe there are a few more where testing individual
>> test-suites
>>>>>>>>>> and/or
>>>>>>>>>>> test-cases are dependent on data loaded in other test-suites
>> and/or
>>>>>>>>>> other
>>>>>>>>>>> test-cases.
>>>>>>>>>> I have the same experience.  Moreover another source of fragility
>> is
>>>>>>>>>> that tests depend on other tests within a single OFBiz
>> “test-case”,
>>>>>>>>>> meaning one test can depend on the data produced by another test.
>>>>>>> This
>>>>>>>>>> is acceptable for a “simple-method-test” because the order of
>>>>>>> execution
>>>>>>>>>> is sequential and managed by OFBiz, but this is problematic for
>>>> JUnit
>>>>>>>>>> tests (Groovy, Java) because the order while being deterministic
>>>>>>> depends
>>>>>>>>>> on the arbitrary order imposed by the JVM.
>>>>>>>>>>
>>>>>>>>>> For example I know for a fact that “QuoteTests.groovy” is
>> suffering
>>>>>>> from
>>>>>>>>>> that issue.
>>>>>>>>>>
>>>>>>>>>>> While I don't hear/read about failing testIntegration (except
>> where
>>>>>>>>>> code in
>>>>>>>>>>> the base is faulty, not when test-suites/cases are faulty), I see
>>>>>>>>>> following
>>>>>>>>>>> failures in test executions in OFBiz against jdk11:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>        1. Execution failed for task ':ofbiz --test
>> component=webapp
>>>>>>> --test
>>>>>>>>>>>        suitename=webapptests'.
>>>>>>>>>>>        2. Execution failed for task ':ofbiz --test
>>>> component=accounting
>>>>>>>>>> --test
>>>>>>>>>>>        suitename=invoicetest'.
>>>>>>>>>>>        3. Execution failed for task ':ofbiz --test component=order
>>>>>>> --test
>>>>>>>>>>>        suitename=ordertests'.
>>>>>>>>>>>        4. Execution failed for task ':ofbiz --test
>> component=product
>>>>>>> --test
>>>>>>>>>>>        suitename=producttests'.
>>>>>>>>>>>
>>>>>>>>>>> Do we have these test failing also when doing the test execution
>>>>>>> against
>>>>>>>>>>> jdk8?
>>>>>>>>>>> *Caveat: I recently set this up, so there may still be some
>>>>>>>>>> configuration
>>>>>>>>>>> issues in the jdk11-test setup.. *
>>>>>>>>>> I have just tested the “ordertests” test-suite with Icedtea 3.7
>>>>>>> (jdk-8)
>>>>>>>>>> and it is still failing, so it seems unrelated in that case.
>>>>>>>>>>
>>>>>>>>>> Thanks.
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Mathieu Lirzin
>>>>>>>>>> GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37
>>>>>>>>>>