Behavior of Groovy vs JUnit tests with test data

[Carsten Schinzer]

Hi everyone,


Recently, I did find that test cases actually are much easier to write in Groovy and hence I started doing that, but now I stumble across the fact that some of the Groovy tests seem to find changes applied to entities from previous tests. The behavior is the following:

- I load the test data with instructions given in the tested XML
- I manipulate data in the first test data, e.g. a status on an entity
- when I run the next test case I do not get the originally loaded entity but the manipulated one from the previous test

This is different from JUnit in Java where the entities seem to exist as loaded from the testdata XML with every new test case.
It would obviously mean that I we need to implement my testing in JUnit (Java) rather when we want extensive testing of the services and need to start from the loaded data every time.

Can anyone confirm this behavior of Groovy over JUnit?
Warm regards


Carsten

[Carsten Schinzer]

Hi all,


I did find and try the following from google search:

- wrap all the tests in a class
- tag the class with @RunWith(SpringRunner.class)
- tag every method that manipulates the entity data with @DirtiesContext(classMode = DirtiesContext.ClassMode.AFTER_EACH_TEST_METHOD)

This has NOT been successful, hence a strong indication that running the cases in Groovy will make entity data manipulations to be sticky and not reverted for the subsequent test case run.
I am going to rework my cases to run as JUnit test cases in a Java class.

Warm regards


Carsten

> Am 03.09.2020 um 13:13 schrieb Carsten Schinzer <[hidden email]>:
>
> Hi everyone,
>
>
> Recently, I did find that test cases actually are much easier to write in Groovy and hence I started doing that, but now I stumble across the fact that some of the Groovy tests seem to find changes applied to entities from previous tests. The behavior is the following:
>
> - I load the test data with instructions given in the tested XML
> - I manipulate data in the first test data, e.g. a status on an entity
> - when I run the next test case I do not get the originally loaded entity but the manipulated one from the previous test
>
> This is different from JUnit in Java where the entities seem to exist as loaded from the testdata XML with every new test case.
> It would obviously mean that I we need to implement my testing in JUnit (Java) rather when we want extensive testing of the services and need to start from the loaded data every time.
>
> Can anyone confirm this behavior of Groovy over JUnit?
> Warm regards
>
>
> Carsten
>

[Jacques Le Roux]

Hi Carsten,

Did you have a look at the WIP under https://issues.apache.org/jira/browse/OFBIZ-11232 and how it's done in subtasks?

Jacques

Le 03/09/2020 à 13:43, Carsten Schinzer a écrit :

> Hi all,
>
>
> I did find and try the following from google search:
>
> - wrap all the tests in a class
> - tag the class with @RunWith(SpringRunner.class)
> - tag every method that manipulates the entity data with @DirtiesContext(classMode = DirtiesContext.ClassMode.AFTER_EACH_TEST_METHOD)
>
> This has NOT been successful, hence a strong indication that running the cases in Groovy will make entity data manipulations to be sticky and not reverted for the subsequent test case run.
> I am going to rework my cases to run as JUnit test cases in a Java class.
>
> Warm regards
>
>
> Carsten
>
>> Am 03.09.2020 um 13:13 schrieb Carsten Schinzer <[hidden email]>:
>>
>> Hi everyone,
>>
>>
>> Recently, I did find that test cases actually are much easier to write in Groovy and hence I started doing that, but now I stumble across the fact that some of the Groovy tests seem to find changes applied to entities from previous tests. The behavior is the following:
>>
>> - I load the test data with instructions given in the tested XML
>> - I manipulate data in the first test data, e.g. a status on an entity
>> - when I run the next test case I do not get the originally loaded entity but the manipulated one from the previous test
>>
>> This is different from JUnit in Java where the entities seem to exist as loaded from the testdata XML with every new test case.
>> It would obviously mean that I we need to implement my testing in JUnit (Java) rather when we want extensive testing of the services and need to start from the loaded data every time.
>>
>> Can anyone confirm this behavior of Groovy over JUnit?
>> Warm regards
>>
>>
>> Carsten
>>

[Carsten Schinzer]

Hello Jacques,


Thanks for the pointer.

My issue is that I find the XML test cases not good enough as e.g. the updateCustRequest service has special behaviors when cancelling or adding stories to a custRequest which are currently not covered by the test in XML.
Similar for the transitions between custRequestStatusIds some are forbidden and some are not and the test cases did not cover either.

The issue is really that in Groovy when starting from a given data load, the data is not refreshed/reset after a test case is run and hence I decided to do the testing of services in full and plain JUnit. Others apparently have decided the same.

In the subtasks you pointed me to, the problem is circumvented by creating the data in a common method before running a test against it. So the data load does not occur via the tested location. I wanted to re-use existing test data for custRequest.

If you tell me that the intent is to also get rid of the test data defined in the tested, then I would be on the wrong track and need to change my code to be like the one you pointed me to. Is there any decision made to also get rid of the test data loads when migrating minilang to groovy?

Warm regards


Carsten

PS: I also find it a pity that the JIRA tickets that are distributed are already subtasks in Stories instead of Stories in Epics. It disallows me to track the work done in sufficient detail other than commenting on the subtask. Do you know why that has been done like this? I should possibly rather discuss with Pierre Smits I guess?

> Am 08.09.2020 um 11:54 schrieb Jacques Le Roux <[hidden email]>:
>
> Hi Carsten,
>
> Did you have a look at the WIP under https://issues.apache.org/jira/browse/OFBIZ-11232 and how it's done in subtasks?
>
> Jacques
>
> Le 03/09/2020 à 13:43, Carsten Schinzer a écrit :
>> Hi all,
>>
>>
>> I did find and try the following from google search:
>>
>> - wrap all the tests in a class
>> - tag the class with @RunWith(SpringRunner.class)
>> - tag every method that manipulates the entity data with @DirtiesContext(classMode = DirtiesContext.ClassMode.AFTER_EACH_TEST_METHOD)
>>
>> This has NOT been successful, hence a strong indication that running the cases in Groovy will make entity data manipulations to be sticky and not reverted for the subsequent test case run.
>> I am going to rework my cases to run as JUnit test cases in a Java class.
>>
>> Warm regards
>>
>>
>> Carsten
>>
>>> Am 03.09.2020 um 13:13 schrieb Carsten Schinzer <[hidden email]>:
>>>
>>> Hi everyone,
>>>
>>>
>>> Recently, I did find that test cases actually are much easier to write in Groovy and hence I started doing that, but now I stumble across the fact that some of the Groovy tests seem to find changes applied to entities from previous tests. The behavior is the following:
>>>
>>> - I load the test data with instructions given in the tested XML
>>> - I manipulate data in the first test data, e.g. a status on an entity
>>> - when I run the next test case I do not get the originally loaded entity but the manipulated one from the previous test
>>>
>>> This is different from JUnit in Java where the entities seem to exist as loaded from the testdata XML with every new test case.
>>> It would obviously mean that I we need to implement my testing in JUnit (Java) rather when we want extensive testing of the services and need to start from the loaded data every time.
>>>
>>> Can anyone confirm this behavior of Groovy over JUnit?
>>> Warm regards
>>>
>>>
>>> Carsten
>>>

[Jacques Le Roux]

Hi Carsten,

Inline...

Le 08/09/2020 à 12:46, Carsten Schinzer a écrit :
> Hello Jacques,
>
>
> Thanks for the pointer.
>
> My issue is that I find the XML test cases not good enough as e.g. the updateCustRequest service has special behaviors when cancelling or adding stories to a custRequest which are currently not covered by the test in XML.
> Similar for the transitions between custRequestStatusIds some are forbidden and some are not and the test cases did not cover either.

+1 for improvement (did not review)


> The issue is really that in Groovy when starting from a given data load, the data is not refreshed/reset after a test case is run and hence I decided to do the testing of services in full and plain JUnit. Others apparently have decided the same.

I guess those persons did that long ago and not related to the move from Minilang to Groovy


> In the subtasks you pointed me to, the problem is circumvented by creating the data in a common method before running a test against it. So the data load does not occur via the tested location. I wanted to re-use existing test data for custRequest.

I was not aware of the issue refreshed/reset after a test case. Indeed we don't use that in Goovy test. Did you have a look at
https://issues.apache.org/jira/browse/OFBIZ-9996 and http://groovy-lang.org/testing.html?


> If you tell me that the intent is to also get rid of the test data defined in the tested, then I would be on the wrong track and need to change my code to be like the one you pointed me to. Is there any decision made to also get rid of the test data loads when migrating minilang to groovy?

I don't think a such decision has been made. Nicolas started with OFBIZ-9996. Since then few improvement has been made, but nothing related to what
you are looking for.


> Warm regards
>
> Carsten
>
> PS: I also find it a pity that the JIRA tickets that are distributed are already subtasks in Stories instead of Stories in Epics. It disallows me to track the work done in sufficient detail other than commenting on the subtask. Do you know why that has been done like this? I should possibly rather discuss with Pierre Smits I guess?

We don't follow Scrum in Jira. We don't use Stories and Epics. We simply create tasks to be done. Sometimes subtasks are an easy way to group the
actions, that's it. I don't know about Pierre's availability, what would you want to discuss exactly?

HTH

Jacques

>
>
>> Am 08.09.2020 um 11:54 schrieb Jacques Le Roux <[hidden email]>:
>>
>> Hi Carsten,
>>
>> Did you have a look at the WIP under https://issues.apache.org/jira/browse/OFBIZ-11232 and how it's done in subtasks?
>>
>> Jacques
>>
>> Le 03/09/2020 à 13:43, Carsten Schinzer a écrit :
>>> Hi all,
>>>
>>>
>>> I did find and try the following from google search:
>>>
>>> - wrap all the tests in a class
>>> - tag the class with @RunWith(SpringRunner.class)
>>> - tag every method that manipulates the entity data with @DirtiesContext(classMode = DirtiesContext.ClassMode.AFTER_EACH_TEST_METHOD)
>>>
>>> This has NOT been successful, hence a strong indication that running the cases in Groovy will make entity data manipulations to be sticky and not reverted for the subsequent test case run.
>>> I am going to rework my cases to run as JUnit test cases in a Java class.
>>>
>>> Warm regards
>>>
>>>
>>> Carsten
>>>
>>>> Am 03.09.2020 um 13:13 schrieb Carsten Schinzer <[hidden email]>:
>>>>
>>>> Hi everyone,
>>>>
>>>>
>>>> Recently, I did find that test cases actually are much easier to write in Groovy and hence I started doing that, but now I stumble across the fact that some of the Groovy tests seem to find changes applied to entities from previous tests. The behavior is the following:
>>>>
>>>> - I load the test data with instructions given in the tested XML
>>>> - I manipulate data in the first test data, e.g. a status on an entity
>>>> - when I run the next test case I do not get the originally loaded entity but the manipulated one from the previous test
>>>>
>>>> This is different from JUnit in Java where the entities seem to exist as loaded from the testdata XML with every new test case.
>>>> It would obviously mean that I we need to implement my testing in JUnit (Java) rather when we want extensive testing of the services and need to start from the loaded data every time.
>>>>
>>>> Can anyone confirm this behavior of Groovy over JUnit?
>>>> Warm regards
>>>>
>>>>
>>>> Carsten
>>>>