[jira] Created: (OFBIZ-3663) Re-executing the tests cause unit test failures

[Nicolas Malin (Jira)]

Re-executing the tests cause unit test failures
-----------------------------------------------

                 Key: OFBIZ-3663
                 URL: https://issues.apache.org/jira/browse/OFBIZ-3663
             Project: OFBiz
          Issue Type: Bug
          Components: accounting
    Affects Versions: SVN trunk
            Reporter: Bob Morley
             Fix For: SVN trunk


After a successful unit test run (from completely empty derby database) I attempted a re-execution of the tests after some programming changes.  A number of tests started to fail (I think 4) which I believe were in the Accounting area (but I could be wrong on that).  Test execution ideally should cause no state change to the database, but at a minimum should not fail if executed more than one time.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[Nicolas Malin (Jira)]

    [ https://issues.apache.org/jira/browse/OFBIZ-3663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12858095#action_12858095 ]

Scott Gray commented on OFBIZ-3663:
-----------------------------------

This is a known problem that I haven't gotten around to dealing with yet.  The problem is that (persisted?) async services can't be rolled back because the JobManager uses the original delegator instead of the temporary test delegator.

I'll have another look when I get a chance, I'm hoping that the changes that David made for multi-tenanting may actually make fixing this problem easier.

> Re-executing the tests cause unit test failures
> -----------------------------------------------
>
>                 Key: OFBIZ-3663
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-3663
>             Project: OFBiz
>          Issue Type: Bug
>          Components: accounting
>    Affects Versions: SVN trunk
>            Reporter: Bob Morley
>             Fix For: SVN trunk
>
>
> After a successful unit test run (from completely empty derby database) I attempted a re-execution of the tests after some programming changes.  A number of tests started to fail (I think 4) which I believe were in the Accounting area (but I could be wrong on that).  Test execution ideally should cause no state change to the database, but at a minimum should not fail if executed more than one time.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

[Nicolas Malin (Jira)]

    [ https://issues.apache.org/jira/browse/OFBIZ-3663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12858556#action_12858556 ]

Bob Morley commented on OFBIZ-3663:
-----------------------------------

I am not convinced this is the problem at all -- the one failure I would point to is EntityTestSuite.testMakeValue which creates four "TestingType" entities and executes to get exactly 4 back.  The trouble is that the "test data files" in service/testdef/data also create "TestingType" entities in the database and they do not appear to be doing a rollback.  So a couple of things --

a) Tests only work because (by happenstance) the entity test suite runs before the services related tests based on how the testdefs are loaded in.
b) On subsequent run, the entity test suite now gets 13 "TestingTypes" the 9 provided by the services and its own set of 4

The short-term solution would be to make the entity test more intelligent.  (aka find how many are there pre-create, do your create, make sure you have 4 more than what you started with).

A different solution would be to enhance "servicetests" such that the "<entity-xml action="load" entity-xml-url="component://service/testdef/data/ServiceTestData.xml"/>" can have an equivalent "finally" step with action "unload" or something similar to clean these things up.

But in general, I think it is a better practice to have a self-contained unit test that is well behaved typically cleaning up after itself via a rollback.  This is different from integration or acceptance tests that would potentially build on each other.

> Re-executing the tests cause unit test failures
> -----------------------------------------------
>
>                 Key: OFBIZ-3663
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-3663
>             Project: OFBiz
>          Issue Type: Bug
>          Components: accounting
>    Affects Versions: SVN trunk
>            Reporter: Bob Morley
>             Fix For: SVN trunk
>
>
> After a successful unit test run (from completely empty derby database) I attempted a re-execution of the tests after some programming changes.  A number of tests started to fail (I think 4) which I believe were in the Accounting area (but I could be wrong on that).  Test execution ideally should cause no state change to the database, but at a minimum should not fail if executed more than one time.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[Nicolas Malin (Jira)]

    [ https://issues.apache.org/jira/browse/OFBIZ-3663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12858711#action_12858711 ]

Scott Gray commented on OFBIZ-3663:
-----------------------------------

Turns out you're right, the first thing the EntitySaxReader does is to clone the delegator which turns off the rollback feature.  I've got a fix for that which I'll get tested and committed soonish.

> Re-executing the tests cause unit test failures
> -----------------------------------------------
>
>                 Key: OFBIZ-3663
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-3663
>             Project: OFBiz
>          Issue Type: Bug
>          Components: accounting
>    Affects Versions: SVN trunk
>            Reporter: Bob Morley
>             Fix For: SVN trunk
>
>
> After a successful unit test run (from completely empty derby database) I attempted a re-execution of the tests after some programming changes.  A number of tests started to fail (I think 4) which I believe were in the Accounting area (but I could be wrong on that).  Test execution ideally should cause no state change to the database, but at a minimum should not fail if executed more than one time.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[Nicolas Malin (Jira)]

     [ https://issues.apache.org/jira/browse/OFBIZ-3663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Scott Gray reassigned OFBIZ-3663:
---------------------------------

    Assignee: Scott Gray

> Re-executing the tests cause unit test failures
> -----------------------------------------------
>
>                 Key: OFBIZ-3663
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-3663
>             Project: OFBiz
>          Issue Type: Bug
>          Components: accounting
>    Affects Versions: SVN trunk
>            Reporter: Bob Morley
>            Assignee: Scott Gray
>             Fix For: SVN trunk
>
>
> After a successful unit test run (from completely empty derby database) I attempted a re-execution of the tests after some programming changes.  A number of tests started to fail (I think 4) which I believe were in the Accounting area (but I could be wrong on that).  Test execution ideally should cause no state change to the database, but at a minimum should not fail if executed more than one time.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[Nicolas Malin (Jira)]

     [ https://issues.apache.org/jira/browse/OFBIZ-3663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Scott Gray closed OFBIZ-3663.
-----------------------------

    Resolution: Fixed

Thanks for the report Bob, fixed in r935869.  Thanks for not taking my word on the cause as well, I'm a bit annoyed at myself for assuming and then putting up with that bug for so long because I thought it was too complicated to fix.

While testing that I realized that entity ECA services weren't being rolled back (they used to work) which was also causing subsequent runs to fail, fixed that in r935870.

> Re-executing the tests cause unit test failures
> -----------------------------------------------
>
>                 Key: OFBIZ-3663
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-3663
>             Project: OFBiz
>          Issue Type: Bug
>          Components: accounting
>    Affects Versions: SVN trunk
>            Reporter: Bob Morley
>            Assignee: Scott Gray
>             Fix For: SVN trunk
>
>
> After a successful unit test run (from completely empty derby database) I attempted a re-execution of the tests after some programming changes.  A number of tests started to fail (I think 4) which I believe were in the Accounting area (but I could be wrong on that).  Test execution ideally should cause no state change to the database, but at a minimum should not fail if executed more than one time.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.