Hello,
Two PRs have been created to address this ticket - please see comments in jira. The first PR is mostly a documentation change as it turned out the required functionality already existed. The second PR introduces some unit testing, but to be able to test MacroFormRenderer without an explosion of mockito mocks, and lots of careful setup in the environment, which would lead to brittle tests, I felt it appropriate to introduce JMockit. How does the development community feel about the trade-off of introducing another test library versus the ability to create tests without a massive amount of, in my opinion, brittle code. Possibly the preferred approach would have been to apply some refactoring to MacroFormRender to allow easier testing. Specifically MFR could be split into two or more classes to: - convert from the field type to the rendered FTL macro string - execute a rendered FTL macro string The refactoring is probably preferable in the long run, but felt too large a task for this particular ticket. What do you think? Can we permit the JMockit tests until some refactoring is carried out? Thanks, Dan. -- Daniel Watford |
Hello,
I've replaced the original PRs with a few more focused ones: https://github.com/apache/ofbiz-framework/pull/24 https://github.com/apache/ofbiz-framework/pull/25 https://github.com/apache/ofbiz-framework/pull/26 Hopefully the above are considered low risk and/or minor improvements and can be committed. The 4th PR: https://github.com/apache/ofbiz-framework/pull/27 covers the introduction of JMockit needed to test the MacroFormRenderer as described in my previous email. Thanks, Dan. On Sat, 22 Feb 2020 at 18:32, Daniel Watford <[hidden email]> wrote: > Hello, > > Two PRs have been created to address this ticket - please see comments in > jira. > > The first PR is mostly a documentation change as it turned out the > required functionality already existed. > > The second PR introduces some unit testing, but to be able to test > MacroFormRenderer without an explosion of mockito mocks, and lots of > careful setup in the environment, which would lead to brittle tests, I felt > it appropriate to introduce JMockit. > > How does the development community feel about the trade-off of introducing > another test library versus the ability to create tests without a massive > amount of, in my opinion, brittle code. > > Possibly the preferred approach would have been to apply some refactoring > to MacroFormRender to allow easier testing. Specifically MFR could be split > into two or more classes to: > - convert from the field type to the rendered FTL macro string > - execute a rendered FTL macro string > > The refactoring is probably preferable in the long run, but felt too large > a task for this particular ticket. > > What do you think? Can we permit the JMockit tests until some refactoring > is carried out? > > Thanks, > > Dan. > > -- > Daniel Watford > -- Daniel Watford |
Hello,
There are two outstanding PRs on OFBIZ-4035. https://github.com/apache/ofbiz-framework/pull/26 should be considered low risk as it completes a minor missing piece of functionality for generating the DOM element id for container fields created in form widgets. https://github.com/apache/ofbiz-framework/pull/27 introduces JMockit in order to unit test MacroFormRenderer which I mentioned in my first email in this thread. I'd like to use JMockit for these hard-to-test cases where code design doesn't lend well to use of libraries like Mockito. We could consider use of JMockit as an indicator that a class might benefit from some architectural refactoring, but at least we'd have some tests in place to help us ensure behaviour is not broken. Thanks, Dan. On Sun, 23 Feb 2020 at 18:54, Daniel Watford <[hidden email]> wrote: > Hello, > > I've replaced the original PRs with a few more focused ones: > > https://github.com/apache/ofbiz-framework/pull/24 > https://github.com/apache/ofbiz-framework/pull/25 > https://github.com/apache/ofbiz-framework/pull/26 > > Hopefully the above are considered low risk and/or minor improvements and > can be committed. > > The 4th PR: > https://github.com/apache/ofbiz-framework/pull/27 > covers the introduction of JMockit needed to test the MacroFormRenderer as > described in my previous email. > > Thanks, > > Dan. > > On Sat, 22 Feb 2020 at 18:32, Daniel Watford <[hidden email]> wrote: > >> Hello, >> >> Two PRs have been created to address this ticket - please see comments in >> jira. >> >> The first PR is mostly a documentation change as it turned out the >> required functionality already existed. >> >> The second PR introduces some unit testing, but to be able to test >> MacroFormRenderer without an explosion of mockito mocks, and lots of >> careful setup in the environment, which would lead to brittle tests, I felt >> it appropriate to introduce JMockit. >> >> How does the development community feel about the trade-off of >> introducing another test library versus the ability to create tests without >> a massive amount of, in my opinion, brittle code. >> >> Possibly the preferred approach would have been to apply some refactoring >> to MacroFormRender to allow easier testing. Specifically MFR could be split >> into two or more classes to: >> - convert from the field type to the rendered FTL macro string >> - execute a rendered FTL macro string >> >> The refactoring is probably preferable in the long run, but felt too >> large a task for this particular ticket. >> >> What do you think? Can we permit the JMockit tests until some refactoring >> is carried out? >> >> Thanks, >> >> Dan. >> >> -- >> Daniel Watford >> > > > -- > Daniel Watford > -- Daniel Watford |
Hi Dan,
why would you prefer JMockit over Mockito? Thanks, Michael Brohl ecomify GmbH - www.ecomify.de Am 02.03.20 um 11:23 schrieb Daniel Watford: > Hello, > > There are two outstanding PRs on OFBIZ-4035. > > https://github.com/apache/ofbiz-framework/pull/26 should be considered low > risk as it completes a minor missing piece of functionality for generating > the DOM element id for container fields created in form widgets. > > https://github.com/apache/ofbiz-framework/pull/27 introduces JMockit in > order to unit test MacroFormRenderer which I mentioned in my first email in > this thread. I'd like to use JMockit for these hard-to-test cases where > code design doesn't lend well to use of libraries like Mockito. > > We could consider use of JMockit as an indicator that a class might benefit > from some architectural refactoring, but at least we'd have some tests in > place to help us ensure behaviour is not broken. > > Thanks, > > Dan. > > On Sun, 23 Feb 2020 at 18:54, Daniel Watford <[hidden email]> wrote: > >> Hello, >> >> I've replaced the original PRs with a few more focused ones: >> >> https://github.com/apache/ofbiz-framework/pull/24 >> https://github.com/apache/ofbiz-framework/pull/25 >> https://github.com/apache/ofbiz-framework/pull/26 >> >> Hopefully the above are considered low risk and/or minor improvements and >> can be committed. >> >> The 4th PR: >> https://github.com/apache/ofbiz-framework/pull/27 >> covers the introduction of JMockit needed to test the MacroFormRenderer as >> described in my previous email. >> >> Thanks, >> >> Dan. >> >> On Sat, 22 Feb 2020 at 18:32, Daniel Watford <[hidden email]> wrote: >> >>> Hello, >>> >>> Two PRs have been created to address this ticket - please see comments in >>> jira. >>> >>> The first PR is mostly a documentation change as it turned out the >>> required functionality already existed. >>> >>> The second PR introduces some unit testing, but to be able to test >>> MacroFormRenderer without an explosion of mockito mocks, and lots of >>> careful setup in the environment, which would lead to brittle tests, I felt >>> it appropriate to introduce JMockit. >>> >>> How does the development community feel about the trade-off of >>> introducing another test library versus the ability to create tests without >>> a massive amount of, in my opinion, brittle code. >>> >>> Possibly the preferred approach would have been to apply some refactoring >>> to MacroFormRender to allow easier testing. Specifically MFR could be split >>> into two or more classes to: >>> - convert from the field type to the rendered FTL macro string >>> - execute a rendered FTL macro string >>> >>> The refactoring is probably preferable in the long run, but felt too >>> large a task for this particular ticket. >>> >>> What do you think? Can we permit the JMockit tests until some refactoring >>> is carried out? >>> >>> Thanks, >>> >>> Dan. >>> >>> -- >>> Daniel Watford >>> >> >> -- >> Daniel Watford >> > smime.p7s (5K) Download Attachment |
Hi Michael,
I don't have a preference between the two mocking libraries, but creating unit tests for MacroFormRenderer necessitates mocking static methods. Mockito doesn't offer static method mocking, but JMockit does. There are work arounds, but I think will be too cumbersome and brittle to be effective. Quoting myself [1]: The second PR extends the first PR and introduces unit tests. The potentially controversial issue is that to implement the tests I felt it was necessary to mock out static methods, functionality that's not available from Mockito, and therefore brought in a test dependency on JMockit. To provide unit tests for MacroFormRenderer without using a mocking library like JMockit will require the class to be refactored which I deemed a bit of a risky change at this point, particularly without existing tests to prove correct behaviour is maintained. Choosing JMockit here was about using the right tool for the job. As far as I know other mocking libraries, such as PowerMock, offer static mocking too but my experience is with JMockit. [1] https://issues.apache.org/jira/browse/OFBIZ-4035?focusedCommentId=17042669&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17042669 Thanks, Dan. On Mon, 2 Mar 2020 at 13:02, Michael Brohl <[hidden email]> wrote: > Hi Dan, > > why would you prefer JMockit over Mockito? > > Thanks, > > Michael Brohl > > ecomify GmbH - www.ecomify.de > > > Am 02.03.20 um 11:23 schrieb Daniel Watford: > > Hello, > > > > There are two outstanding PRs on OFBIZ-4035. > > > > https://github.com/apache/ofbiz-framework/pull/26 should be considered > low > > risk as it completes a minor missing piece of functionality for > generating > > the DOM element id for container fields created in form widgets. > > > > https://github.com/apache/ofbiz-framework/pull/27 introduces JMockit in > > order to unit test MacroFormRenderer which I mentioned in my first email > in > > this thread. I'd like to use JMockit for these hard-to-test cases where > > code design doesn't lend well to use of libraries like Mockito. > > > > We could consider use of JMockit as an indicator that a class might > benefit > > from some architectural refactoring, but at least we'd have some tests in > > place to help us ensure behaviour is not broken. > > > > Thanks, > > > > Dan. > > > > On Sun, 23 Feb 2020 at 18:54, Daniel Watford <[hidden email]> wrote: > > > >> Hello, > >> > >> I've replaced the original PRs with a few more focused ones: > >> > >> https://github.com/apache/ofbiz-framework/pull/24 > >> https://github.com/apache/ofbiz-framework/pull/25 > >> https://github.com/apache/ofbiz-framework/pull/26 > >> > >> Hopefully the above are considered low risk and/or minor improvements > and > >> can be committed. > >> > >> The 4th PR: > >> https://github.com/apache/ofbiz-framework/pull/27 > >> covers the introduction of JMockit needed to test the MacroFormRenderer > as > >> described in my previous email. > >> > >> Thanks, > >> > >> Dan. > >> > >> On Sat, 22 Feb 2020 at 18:32, Daniel Watford <[hidden email]> wrote: > >> > >>> Hello, > >>> > >>> Two PRs have been created to address this ticket - please see comments > in > >>> jira. > >>> > >>> The first PR is mostly a documentation change as it turned out the > >>> required functionality already existed. > >>> > >>> The second PR introduces some unit testing, but to be able to test > >>> MacroFormRenderer without an explosion of mockito mocks, and lots of > >>> careful setup in the environment, which would lead to brittle tests, I > felt > >>> it appropriate to introduce JMockit. > >>> > >>> How does the development community feel about the trade-off of > >>> introducing another test library versus the ability to create tests > without > >>> a massive amount of, in my opinion, brittle code. > >>> > >>> Possibly the preferred approach would have been to apply some > refactoring > >>> to MacroFormRender to allow easier testing. Specifically MFR could be > split > >>> into two or more classes to: > >>> - convert from the field type to the rendered FTL macro string > >>> - execute a rendered FTL macro string > >>> > >>> The refactoring is probably preferable in the long run, but felt too > >>> large a task for this particular ticket. > >>> > >>> What do you think? Can we permit the JMockit tests until some > refactoring > >>> is carried out? > >>> > >>> Thanks, > >>> > >>> Dan. > >>> > >>> -- > >>> Daniel Watford > >>> > >> > >> -- > >> Daniel Watford > >> > > > > -- Daniel Watford |
Hello Daniel,
Daniel Watford <[hidden email]> writes: > I don't have a preference between the two mocking libraries, but creating > unit tests for MacroFormRenderer necessitates mocking static methods. > Mockito doesn't offer static method mocking, but JMockit does. > [...] > To provide unit tests for MacroFormRenderer without using a mocking library > like JMockit will require the class to be refactored which I deemed a bit > of a risky change at this point, particularly without existing tests to > prove correct behaviour is maintained. Any effort that take into account how things can effectively be tested is a good thing for OFBiz. :-) My impression is that the "need" to mock static methods is a symptom of entangled code. And that making the code simpler and better designed will remove that need. As you said having to refactor without tests in the first place is not good but usage of advanced mocking techniques to workaround the fact that basic unit testing is too hard is not ideal neither. Another solution (that is not ideal neither) is to rely on manual testing in the first place (or automatic integration tests if you have time) introduce the architectural refactoring and then add you unit test with explicit arguments based dependency injection instead of mocks. In any case that is a matter of compromise. :-) -- Mathieu Lirzin GPG: F2A3 8D7E EB2B 6640 5761 070D 0ADE E100 9460 4D37 |
Hi Mathieu,
Yes I agree that mocking of static methods is an indicator of refactoring needed. There are always exceptions to every rule, but in this case with MacroFormRenderer longer than 3000 lines in R18 and its role is to map from ModelFormField instances to string representation of FTL macro calls, and then to execute those FTL macros, it feels like an excellent candidate for a bit of refactoring. However, to do so without any sort of test framework in place will be introducing risk. At first a refactoring effort might seem simple, but with such a large class we are likely to find some difficulties. Therefore if I had a vote I would suggest accepting the JMockit based tests with a view to retire and replace them with more standard and recognisable Mockito tests once made possible through refactoring. If the committers accept https://github.com/apache/ofbiz-framework/pull/26 and https://github.com/apache/ofbiz-framework/pull/27 I will be happy to open and work on a ticket to refactor MacroFormRenderer and associated tests. I'd aim to deliver any refactoring as a number of small patches/PRs over a period of time. Thanks, Dan. On Mon, 2 Mar 2020 at 16:17, Mathieu Lirzin <[hidden email]> wrote: > Hello Daniel, > > Daniel Watford <[hidden email]> writes: > > > I don't have a preference between the two mocking libraries, but creating > > unit tests for MacroFormRenderer necessitates mocking static methods. > > Mockito doesn't offer static method mocking, but JMockit does. > > [...] > > To provide unit tests for MacroFormRenderer without using a mocking > library > > like JMockit will require the class to be refactored which I deemed a bit > > of a risky change at this point, particularly without existing tests to > > prove correct behaviour is maintained. > > Any effort that take into account how things can effectively be tested > is a good thing for OFBiz. :-) > > My impression is that the "need" to mock static methods is a symptom of > entangled code. And that making the code simpler and better designed > will remove that need. As you said having to refactor without tests in > the first place is not good but usage of advanced mocking techniques to > workaround the fact that basic unit testing is too hard is not ideal > neither. > > Another solution (that is not ideal neither) is to rely on manual > testing in the first place (or automatic integration tests if you have > time) introduce the architectural refactoring and then add you unit test > with explicit arguments based dependency injection instead of mocks. > > In any case that is a matter of compromise. :-) > > -- > Mathieu Lirzin > GPG: F2A3 8D7E EB2B 6640 5761 070D 0ADE E100 9460 4D37 > -- Daniel Watford |
+1
On 2020/03/02 16:52:08 Daniel Watford wrote: > Hi Mathieu, > > Yes I agree that mocking of static methods is an indicator of refactoring > needed. > > There are always exceptions to every rule, but in this case with > MacroFormRenderer longer than > 3000 lines in R18 and its role is to map from ModelFormField instances to > string representation > of FTL macro calls, and then to execute those FTL macros, it feels like an > excellent candidate for a > bit of refactoring. > > However, to do so without any sort of test framework in place will be > introducing risk. > > At first a refactoring effort might seem simple, but with such a large > class we are likely to find > some difficulties. Therefore if I had a vote I would suggest accepting the > JMockit based tests with > a view to retire and replace them with more standard and recognisable > Mockito tests once > made possible through refactoring. > > If the committers accept https://github.com/apache/ofbiz-framework/pull/26 > and > https://github.com/apache/ofbiz-framework/pull/27 I will be happy to open > and work on a ticket to > refactor MacroFormRenderer and associated tests. I'd aim to deliver any > refactoring as a number > of small patches/PRs over a period of time. > > Thanks, > > Dan. > > On Mon, 2 Mar 2020 at 16:17, Mathieu Lirzin <[hidden email]> > wrote: > > > Hello Daniel, > > > > Daniel Watford <[hidden email]> writes: > > > > > I don't have a preference between the two mocking libraries, but creating > > > unit tests for MacroFormRenderer necessitates mocking static methods. > > > Mockito doesn't offer static method mocking, but JMockit does. > > > [...] > > > To provide unit tests for MacroFormRenderer without using a mocking > > library > > > like JMockit will require the class to be refactored which I deemed a bit > > > of a risky change at this point, particularly without existing tests to > > > prove correct behaviour is maintained. > > > > Any effort that take into account how things can effectively be tested > > is a good thing for OFBiz. :-) > > > > My impression is that the "need" to mock static methods is a symptom of > > entangled code. And that making the code simpler and better designed > > will remove that need. As you said having to refactor without tests in > > the first place is not good but usage of advanced mocking techniques to > > workaround the fact that basic unit testing is too hard is not ideal > > neither. > > > > Another solution (that is not ideal neither) is to rely on manual > > testing in the first place (or automatic integration tests if you have > > time) introduce the architectural refactoring and then add you unit test > > with explicit arguments based dependency injection instead of mocks. > > > > In any case that is a matter of compromise. :-) > > > > -- > > Mathieu Lirzin > > GPG: F2A3 8D7E EB2B 6640 5761 070D 0ADE E100 9460 4D37 > > > > > -- > Daniel Watford > |
In reply to this post by Daniel Watford
Daniel Watford <[hidden email]> writes:
> Yes I agree that mocking of static methods is an indicator of refactoring > needed. > > There are always exceptions to every rule, but in this case with > MacroFormRenderer longer than > 3000 lines in R18 and its role is to map from ModelFormField instances to > string representation > of FTL macro calls, and then to execute those FTL macros, it feels like an > excellent candidate for a > bit of refactoring. I feel the pain. > However, to do so without any sort of test framework in place will be > introducing risk. You are right. If you are fearful of the code you touch (which is legitimate in this case) indeed adding some test automation is a nice idea. > At first a refactoring effort might seem simple, but with such a large > class we are likely to find some difficulties. Therefore if I had a > vote I would suggest accepting the JMockit based tests AFAIC I think the unbearable complexity and awfulness of the implementation FTL/Screen rendering justifies adopting complex testing techniques and adding a new even partially redundant dependency like JMockit, even if this is sad to have to do so. PS: Votes in regards code modification in a community with PMC members that basically disagree on everything else than “We are an awesome community” is not a solution to anything because any veto with a valid technical decision like for example “JMockit is redundant with Mockito purpose which is already used” would simply block your proposition, so a vote for code modification serves only to validate a consensus. [1] [1] https://www.apache.org/foundation/voting.html#votes-on-code-modification > with a view to retire and replace them with more standard and > recognisable Mockito tests once made possible through refactoring. I would suggest seeking the goal of adopting basic dependency injection with explicit method/constructor arguments which leads to simpler tests and is a sign of well designed code, instead of seeking the replacement of advanced mocking techniques with less advanced ones. Mocking is sometimes useful but should be avoided when possible. > If the committers accept https://github.com/apache/ofbiz-framework/pull/26 > and > https://github.com/apache/ofbiz-framework/pull/27 I will be happy to open > and work on a ticket to > refactor MacroFormRenderer and associated tests. I'd aim to deliver any > refactoring as a number > of small patches/PRs over a period of time. Good luck. -- Mathieu Lirzin GPG: F2A3 8D7E EB2B 6640 5761 070D 0ADE E100 9460 4D37 |
Hi Mathieu,
Apologies if gmail mess up email quoting. On Thu, 5 Mar 2020 at 10:33, Mathieu Lirzin <[hidden email]> wrote: > > PS: Votes in regards code modification in a community with PMC members > that basically disagree on everything else than “We are an awesome > community” is not a solution to anything because any veto with a valid > technical decision like for example “JMockit is redundant with Mockito > purpose which is already used” would simply block your proposition, so a > vote for code modification serves only to validate a consensus. [1] > > [1] > https://www.apache.org/foundation/voting.html#votes-on-code-modification > > into how decisions get accepted. We already have a +1 from James. Can I count on your vote, Mathieu? :) Regarding the example argument “JMockit is redundant with Mockito purpose which is already used”: I hope I've demonstrated in this thread that Mockito is not fit for purpose in this particular case due to the architecture of the MacroFormRenderer class; and that refactoring MacroFormRenderer without the support of tests is risky and therefore a worse proposition that introducing JMockit. Therefore a veto with that particular argument shouldn't be considered valid. Also, since we are only discussing test code, we can easily remove them later if sentiment goes against JMockit. > > with a view to retire and replace them with more standard and > > recognisable Mockito tests once made possible through refactoring. > > I would suggest seeking the goal of adopting basic dependency injection > with explicit method/constructor arguments which leads to simpler tests > and is a sign of well designed code, instead of seeking the replacement > of advanced mocking techniques with less advanced ones. Mocking is > sometimes useful but should be avoided when possible. > I'd happily support this and leave the JMockit tests in place if that is considered reasonable after refactoring the class to introduce dependency injection. But I also wouldn't stand in the way of later conversion to Mockito if the community wanted it. In summary, I'm asking the community to accept introduction of a new mocking library to support a longer term piece of refactoring work. Committers! Bring me your +1 and lets get these PRs merged :) Thanks, Dan. -- Daniel Watford |
Administrator
|
Le 05/03/2020 à 12:07, Daniel Watford a écrit :
> In summary, I'm asking the community to accept introduction of a new > mocking library to support a longer term piece of refactoring work. > > Committers! Bring me your +1 and lets get these PRs merged :) I'm actually focused on something else, but I intend to review :) Of course all reviews and votes (on Jira) would be appreciated... Jacques |
Administrator
|
Le 05/03/2020 à 14:18, Jacques Le Roux a écrit :
> Le 05/03/2020 à 12:07, Daniel Watford a écrit : >> In summary, I'm asking the community to accept introduction of a new >> mocking library to support a longer term piece of refactoring work. >> >> Committers! Bring me your +1 and lets get these PRs merged :) > > I'm actually focused on something else, but I intend to review :) > > Of course all reviews and votes (on Jira) would be appreciated... And vote on PRs are as good for me: https://github.com/MechWarriorOnline/issue-tracker/wiki/How-To-Upvote-An-Issue > > Jacques > |
In reply to this post by Daniel Watford
Daniel Watford <[hidden email]> writes:
> We already have a +1 from James. Can I count on your vote, Mathieu? :) You have a +0 on my side. As I already said I support the testing effort because OFBiz need people caring about writing maintainable tests and I found legitimate to introduce an extra mocking library dependency to mitigate the poor testability of current implementation even if it is not ideal. > Regarding the example argument “JMockit is redundant with Mockito purpose > which is already used”: > I hope I've demonstrated in this thread that Mockito is not fit for purpose > in this particular case due to the architecture of the MacroFormRenderer > class; and that refactoring MacroFormRenderer without the support of tests > is risky and therefore a worse proposition that introducing JMockit. > Therefore a veto with that particular argument shouldn't be considered > valid. I don't think this demonstration can effectively prove that this hypothetical technical argument is invalid because “worse” depends on the perspective and software is always about trade-offs meaning that something that is “worse” in some facet is “better” in another. Imagine a potential PMC member who cares more about the size of the artifact footprint than about the actual test coverage. With a bit of insistance combined with a dose of FUD [1] (...customers will drop OFBiz, ...this will introduce dependency incompatibilities) she can easily make it appear as a valid “technical” argument to some doubtful people. So the best you can do is to demonstrate that something is more desirable than some alternative trade-offs with the limit that you have a finite time to spend on that thing and that people can have infinitely many alternatives in mind. Which mean concretely in the presence of stubborn or dishonest people no way to demonstrate anything. ;-) This is going off-topic and since nobody is currently making such kind of arguments so let's drop this discussion. :-) Thanks. [1] https://en.wikipedia.org/wiki/Fear%2C_uncertainty%2C_and_doubt -- Mathieu Lirzin GPG: F2A3 8D7E EB2B 6640 5761 070D 0ADE E100 9460 4D37 |
Free forum by Nabble | Edit this page |