[jira] Created: (OFBIZ-2259) Testing - Rollback database changes after each test-suite using the GenericDelegator

[Nicolas Malin (Jira)]

Testing - Rollback database changes after each test-suite using the GenericDelegator
------------------------------------------------------------------------------------

                 Key: OFBIZ-2259
                 URL: https://issues.apache.org/jira/browse/OFBIZ-2259
             Project: OFBiz
          Issue Type: New Feature
          Components: framework
            Reporter: Scott Gray
            Assignee: Scott Gray


During testing have the delegator keep track of all changes made to the database and then roll them back at the end of each test suite.  This will allow the same demo data to be reused across test suites without having to worry about what changes previous tests have made to the data and solves this problem regardless of the database being used.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[Nicolas Malin (Jira)]

     [ https://issues.apache.org/jira/browse/OFBIZ-2259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Scott Gray updated OFBIZ-2259:
------------------------------

    Attachment: rollback.patch

It would be great to get some feedback on this before I commit it in the next couple of days.

In terms of performance the rollback adds about 15-20% to the time taken to perform all tests which on my machine equates to about 20-30 seconds

> Testing - Rollback database changes after each test-suite using the GenericDelegator
> ------------------------------------------------------------------------------------
>
>                 Key: OFBIZ-2259
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-2259
>             Project: OFBiz
>          Issue Type: New Feature
>          Components: framework
>            Reporter: Scott Gray
>            Assignee: Scott Gray
>         Attachments: rollback.patch
>
>
> During testing have the delegator keep track of all changes made to the database and then roll them back at the end of each test suite.  This will allow the same demo data to be reused across test suites without having to worry about what changes previous tests have made to the data and solves this problem regardless of the database being used.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[Nicolas Malin (Jira)]

    [ https://issues.apache.org/jira/browse/OFBIZ-2259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12693978#action_12693978 ]

David E. Jones commented on OFBIZ-2259:
---------------------------------------

This looks pretty good in general.

One small issue: it is not thread safe. At some point we may decide that running test suites one at a time is too slow. In other words, chances are running a few at a time will result in an over all performance increase compared to running them in serial.

One way we could possibly do this is to clone the delegator and use one instance per test suite. That's the first that comes to mind anyway, there are probably better options... A ThreadLocal variable is something I always consider to be a bit of a hack, but they do have their place. In this case that might not work as a test case could have multiple threads (not sure if any do...).

> Testing - Rollback database changes after each test-suite using the GenericDelegator
> ------------------------------------------------------------------------------------
>
>                 Key: OFBIZ-2259
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-2259
>             Project: OFBiz
>          Issue Type: New Feature
>          Components: framework
>            Reporter: Scott Gray
>            Assignee: Scott Gray
>         Attachments: rollback.patch
>
>
> During testing have the delegator keep track of all changes made to the database and then roll them back at the end of each test suite.  This will allow the same demo data to be reused across test suites without having to worry about what changes previous tests have made to the data and solves this problem regardless of the database being used.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[Nicolas Malin (Jira)]

    [ https://issues.apache.org/jira/browse/OFBIZ-2259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12693980#action_12693980 ]

David E. Jones commented on OFBIZ-2259:
---------------------------------------

Thinking about this more I like the idea of a sort of cloned delegator. In other words instead of doing something like:

modelSuite.getDelegator().setTestMode(true);

we would do something like:

GenericDelegator testDelegator = modelSuite.getDelegator().makeTestDelegator();

and the later on near the end instead of:

modelSuite.getDelegator().performUndo();
modelSuite.getDelegator().setTestMode(false);

maybe something like:

testDelegator.rollback();

> Testing - Rollback database changes after each test-suite using the GenericDelegator
> ------------------------------------------------------------------------------------
>
>                 Key: OFBIZ-2259
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-2259
>             Project: OFBiz
>          Issue Type: New Feature
>          Components: framework
>            Reporter: Scott Gray
>            Assignee: Scott Gray
>         Attachments: rollback.patch
>
>
> During testing have the delegator keep track of all changes made to the database and then roll them back at the end of each test suite.  This will allow the same demo data to be reused across test suites without having to worry about what changes previous tests have made to the data and solves this problem regardless of the database being used.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[Nicolas Malin (Jira)]

    [ https://issues.apache.org/jira/browse/OFBIZ-2259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12694762#action_12694762 ]

Scott Gray commented on OFBIZ-2259:
-----------------------------------

Thanks for the feedback David, those are great points and I'll get them addressed as soon as I get a spare moment.

> Testing - Rollback database changes after each test-suite using the GenericDelegator
> ------------------------------------------------------------------------------------
>
>                 Key: OFBIZ-2259
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-2259
>             Project: OFBiz
>          Issue Type: New Feature
>          Components: framework
>            Reporter: Scott Gray
>            Assignee: Scott Gray
>         Attachments: rollback.patch
>
>
> During testing have the delegator keep track of all changes made to the database and then roll them back at the end of each test suite.  This will allow the same demo data to be reused across test suites without having to worry about what changes previous tests have made to the data and solves this problem regardless of the database being used.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[Nicolas Malin (Jira)]

    [ https://issues.apache.org/jira/browse/OFBIZ-2259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12695703#action_12695703 ]

Jacopo Cappellato commented on OFBIZ-2259:
------------------------------------------

Scott,

looks like a great enhancements, thanks for working at it.
Do you think we could use this work also as a way to prepare demo data for a test?
What I have in mind is something like:
1) I start ofbiz and when I am ready to prepare data I activate the "test mode"
2) I prepare all the data I need
3) instead of just rolling back everything we export all the changes recorded by the delegator in an xml file

The data could then be used to prepare the starting scenario for a use case/test.

Do you think it could work?

Jacopo

> Testing - Rollback database changes after each test-suite using the GenericDelegator
> ------------------------------------------------------------------------------------
>
>                 Key: OFBIZ-2259
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-2259
>             Project: OFBiz
>          Issue Type: New Feature
>          Components: framework
>            Reporter: Scott Gray
>            Assignee: Scott Gray
>         Attachments: rollback.patch
>
>
> During testing have the delegator keep track of all changes made to the database and then roll them back at the end of each test suite.  This will allow the same demo data to be reused across test suites without having to worry about what changes previous tests have made to the data and solves this problem regardless of the database being used.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[Nicolas Malin (Jira)]

    [ https://issues.apache.org/jira/browse/OFBIZ-2259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12695704#action_12695704 ]

Scott Gray commented on OFBIZ-2259:
-----------------------------------

Hi Jacopo

I think it is possible but we couldn't use the new data in regular demo data files due to any updates performed that could break existing unit tests.  We could definitely use the generated data within test suites though:
    <test-case case-name="load-service-test-data">
        <entity-xml action="load" entity-xml-url="component://service/testdef/data/ServiceTestData.xml"/>
    </test-case>
Sounds pretty cool and should be fairly easy to implement, I'll add it to my todo list :-)

> Testing - Rollback database changes after each test-suite using the GenericDelegator
> ------------------------------------------------------------------------------------
>
>                 Key: OFBIZ-2259
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-2259
>             Project: OFBiz
>          Issue Type: New Feature
>          Components: framework
>            Reporter: Scott Gray
>            Assignee: Scott Gray
>         Attachments: rollback.patch
>
>
> During testing have the delegator keep track of all changes made to the database and then roll them back at the end of each test suite.  This will allow the same demo data to be reused across test suites without having to worry about what changes previous tests have made to the data and solves this problem regardless of the database being used.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[Nicolas Malin (Jira)]

    [ https://issues.apache.org/jira/browse/OFBIZ-2259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12695715#action_12695715 ]

Jacques Le Roux commented on OFBIZ-2259:
----------------------------------------

Yes good idea !

> Testing - Rollback database changes after each test-suite using the GenericDelegator
> ------------------------------------------------------------------------------------
>
>                 Key: OFBIZ-2259
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-2259
>             Project: OFBiz
>          Issue Type: New Feature
>          Components: framework
>            Reporter: Scott Gray
>            Assignee: Scott Gray
>         Attachments: rollback.patch
>
>
> During testing have the delegator keep track of all changes made to the database and then roll them back at the end of each test suite.  This will allow the same demo data to be reused across test suites without having to worry about what changes previous tests have made to the data and solves this problem regardless of the database being used.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[Nicolas Malin (Jira)]

     [ https://issues.apache.org/jira/browse/OFBIZ-2259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Scott Gray updated OFBIZ-2259:
------------------------------

    Attachment: rollback.patch

Updated version of rollback functionality.

David, the cloned delegator was a bit more work than I first thought and meant I had to change a few things in various places that could possibly have side effects so it would be great if you could take another look and let me know what you think.

Also, there are currently 3 types of thing that aren't rolled back during testing:
1.  Data loaded through the entity-xml test tag, this is because EntitySaxReader also clones the delegator but I don't why it does that so I've hesitated to change how it works.  I would like to resolve this one before committing though.
2.  Data stored during async service calls
3.  Changes to the SequenceValueItem table, I haven't looked but I'm guessing rolling that back might cause problems and it's not such a big deal anyway.


Regards
Scott

> Testing - Rollback database changes after each test-suite using the GenericDelegator
> ------------------------------------------------------------------------------------
>
>                 Key: OFBIZ-2259
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-2259
>             Project: OFBiz
>          Issue Type: New Feature
>          Components: framework
>            Reporter: Scott Gray
>            Assignee: Scott Gray
>         Attachments: rollback.patch, rollback.patch
>
>
> During testing have the delegator keep track of all changes made to the database and then roll them back at the end of each test suite.  This will allow the same demo data to be reused across test suites without having to worry about what changes previous tests have made to the data and solves this problem regardless of the database being used.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[Nicolas Malin (Jira)]

     [ https://issues.apache.org/jira/browse/OFBIZ-2259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Scott Gray updated OFBIZ-2259:
------------------------------

    Attachment: rollback.patch

Sorry that patch had a few issues, here's a better version

> Testing - Rollback database changes after each test-suite using the GenericDelegator
> ------------------------------------------------------------------------------------
>
>                 Key: OFBIZ-2259
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-2259
>             Project: OFBiz
>          Issue Type: New Feature
>          Components: framework
>            Reporter: Scott Gray
>            Assignee: Scott Gray
>         Attachments: rollback.patch, rollback.patch, rollback.patch
>
>
> During testing have the delegator keep track of all changes made to the database and then roll them back at the end of each test suite.  This will allow the same demo data to be reused across test suites without having to worry about what changes previous tests have made to the data and solves this problem regardless of the database being used.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[Nicolas Malin (Jira)]

     [ https://issues.apache.org/jira/browse/OFBIZ-2259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Scott Gray closed OFBIZ-2259.
-----------------------------

       Resolution: Fixed
    Fix Version/s: SVN trunk

Forgot to close, this was committed in r765057

> Testing - Rollback database changes after each test-suite using the GenericDelegator
> ------------------------------------------------------------------------------------
>
>                 Key: OFBIZ-2259
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-2259
>             Project: OFBiz
>          Issue Type: New Feature
>          Components: framework
>            Reporter: Scott Gray
>            Assignee: Scott Gray
>             Fix For: SVN trunk
>
>         Attachments: rollback.patch, rollback.patch, rollback.patch
>
>
> During testing have the delegator keep track of all changes made to the database and then roll them back at the end of each test suite.  This will allow the same demo data to be reused across test suites without having to worry about what changes previous tests have made to the data and solves this problem regardless of the database being used.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.